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Executive Summary 


The concept of Network-Centric Warfare is rapidly becoming a reality for today’s modem 
military. In practically every operational context—from simulation-based training, to digital 
collaborative planning, to real-time video conferencing on the battlefield—the tools used by the 
modem “digital warrior” are networked and software based. As the digitization of the battlefield 
progresses, tomorrow’s soldiers must be able to adapt, to innovate, and to rapidly transfer digital 
knowledge and skills to new software and new equipment. Soldiers need training in transferable 
digital skills in addition to training in setting-up, using, and maintaining specific digital 
equipment. This paper reports on a project that addresses a critical need for tomorrow’s Army: 
Training in the ability to adapt digital skills to the ever-changing evolution of technology. 

Digital equipment and communication evolve rapidly - the shortened supplier design/production 
cycles and uniquely fast improvements in software and computer components mean that new 
equipment and techniques are fielded with unprecedented speed. The stability of the military 
environment, and the concomitant predictability of tasks and equipment, has become a thing of 
the past. This means that the “half-life” of equipment training is short, so that a soldier may face 
the need to master new equipment in the field. In addition. Army personnel may have to work 
under an intense optempo, under which learning conditions are not optimal. It is not sufficient 
for soldiers simply to learn how to operate today’s digital equipment; there is a need for 
adaptability in learning to use equipment requiring digital skills under difficult conditions. The 
successful warfighters are those who possess a solid foundation of adaptive digital-literacy that 
goes beyond abilities to utilize specific tools and applications, and includes deep knowledge of 
digital concepts that are relevant in every context of use. 

Ensuring this level of knowledge transfer will require new approaches to training, focused on 
providing a deep foundation of digital skills and knowledge, and a set of “tools” and strategies 
that the warfighters can use to apply what they already know to actively engage and learn new 
tools. In fact, one of the major drivers for the Objective Force, Task Force TLS (the “Training,” 
“Leadership,” and “Soldier” parts of DTLOMS), recognizes that the role of the soldier will 
change dramatically because of changes in technology, and that this will result in the need for an 
increase in adaptive skills. 

As outlined in this Phase I report, Aptima and the Group for Organizational Effectiveness (gOE) 
have lain the groundwork for an innovative digital-skills training package designed to increase 
the adaptability of digital skills. The key innovative idea, developed and demonstrated in Phase 
I, is a modular training package that links fundamental digital knowledge to device-specific 
knowledge and to the specific tasks that soldiers are trained to perform. Generalizable, 
transferable digital skills are not taught in a vacuum, but instead are learned in the context of 
device- and job-specific goals. This approach was developed based on the literature on learning 
and transfer of training in procedural domains, which suggests that linking “how to” knowledge 
to domain goals and subgoals helps learners to adapt parts of solutions to new problems. 

Specifically, the training package has two “modules,” one module covers general device 
knowledge and procedures—such as general computer skills, or knowledge, skills, and abilities 
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required for networking computers together—that can be used for any domain in which general 
device knowledge is needed, and the other module trains how to map this general device 
knowledge onto a specific computer system—which requires training focused on a specific 
system. In Phase I, we selected the AFATDS system used by the Army Field Artillery as the 
focus of our training. The general device knowledge training will ensure true adaptability when 
going from one computer system to another, while the device-specific training will allow the 
trainees to take full advantage of the general training in a specific situation. 

This report also outlines the plans for Phase n, where we will expand the training program to 
include all basic digital skills, and improve upon it significantly. Like the Phase I prototype, we 
envision a web-based system that enables both remote hosting and, if required, client or stand¬ 
alone platform installation. It is also anticipated that, unlike the Phase I prototype, the final 
system will be database capable, with administrative capabilities to enable non-HTML proficient 
users to tailor Skills Bridge information for systems other than AFATDS. That is, we expect to 
specify and create a product that has general applicability and is easily customized to provide 
maximum learning regardless of the digital system targeted. 

Concerted commercialization will occur throughout the entire Phase II period. Within the Army, 
the most immediate application is for the Intell school, where soldiers must stay current with 
frequently changing software. For non-DoD government applications, we see the digital skill 
adaptability training supporting the Department of Education’s lifelong learning programs and 
the Department of Labor’s programs designed to improve technology skills among adults. 
Outside the government, this training could be used in any job environment that requires the use 
of technology to ensure the ability of workers to keep up with quickly changing systems. 

Industry is currently facing a severe problem as workers must be retrained after the insertion of 
technological updates—the digital skill adaptability training could mitigate or alleviate this 
problem altogether. 
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1. Introduction 

This SBER project addresses a critical need for tomorrow’s Army: Training in the ability to adapt 
digital skills to the ever-changing evolution of technology. It is not sufficient for soldiers simply 
to learn how to operate today’s digital equipment. As the digitization of the battlefield 
progresses, tomorrow’s soldiers must be able to adapt, to innovate, and to rapidly transfer digital 
knowledge and skills to new software and new equipment. Soldiers need training in transferable 
digital skills in addition to training in setting-up, using, and maintaining specific digital 
equipment. 

In Phase I, Aptima and the Group for Organizational Effectiveness (gOE) laid the groundwork 
for an innovative digital-skills training package designed to increase the adaptability of digital 
skills. Our key innovative idea, developed and demonstrated in Phase I, is a modular training 
package that links fundamental digital knowledge to device-specific knowledge and to the 
specific tasks that soldiers are trained to perform. Generalizable, transferable digital skills are 
not taught in a vacuum, but instead are learned in the context of device- and job-specific goals. 
This approach was developed based on the literature on learning and transfer of training in 
procedural domains, which suggests that linking “how to” knowledge to domain goals and 
subgoals helps learners to adapt parts of solutions to new problems. 

Specifically, the training package has two “modules,” one module covers general device 
knowledge and procedures—such as general computer skills, or knowledge, skills, and abilities 
required for networking computers together—that can be used for any domain in which general 
device knowledge is needed, and the other module trains how to map this general device 
knowledge onto a specific computer system—which requires training focused on a specific 
system. In Phase I, we selected the AFATDS system used by the Army Field Artillery as the 
focus of our training. The general device knowledge training will ensure true adaptability when 
going from one computer system to another, while the device-specific training \yill allow the 
trainees to take full advantage of the general training in a specific situation. 

Training in adaptive digital skills is critical for tomorrow’s Army 

The concept of Network-Centric Warfare is rapidly becoming a reality for today’s modem 
military. In practically every operational context—from simulation-based training, to digital 
collaborative planning, to real-time video conferencing on the battlefield—the tools used by the 
modem “digital warrior” are networked and software based. Digital equipment and 
communication evolve rapidly - the shortened supplier design/production cycles and uniquely 
fast improvements in software and computer components mean that new equipment and 
techniques are fielded with unprecedented speed. The stability of the military environment, and 
the concomitant predictability of tasks and equipment, has become a thing of the past. This 
means that the “half-life” of equipment training is short, so that a soldier may face the need to 
master new equipment in the field. In addition. Army personnel may have to work under an 
intense optempo, under which learning conditions are not optimal. Thus there is a need for 
adaptability in learning to use equipment requiring digital skills under difficult conditions. 

In such digital environments, the successful warfighters are those who possess a solid 
foundation of adaptive digital-literacy that goes beyond abilities to utilize specific tools and 
applications, and includes deep knowledge of digital concepts that are relevant in every context 
of use. Whether faced with new tools, incremental improvements of familiar tools, or the same 
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tools in new contexts (e.g., from a CONUS simulation setting, to a theater of war, or drug ops 
application), the digital warrior will be expected to apply what he or she already knows to 
quickly come up to speed on new tools, often with “just-in-time” rapidity. Ensuring this level of 
knowledge transfer will require new approaches to training, focused on providing a deep 
foundation of digital skills and knowledge, and a set of “tools” and strategies that the warfighters 
can use to apply what they already know to actively engage and learn new tools. In fact, one of 
the major drivers for the Objective Force, Task Force TLS (the “Training,” “Leadership,” and 
“Soldier” parts of DTLOMS), recognizes that the role of the soldier will change dramatically 
because of changes in technology, and that this will result in the need for an increase in adaptive 
skills. 

The U.S. Army Research Institute (ARI) currently has a research program focusing on how to 
best train the digital skills necessary for this battlefield digitization. ARI argues that one of the 
best ways to maximize the benefits of digitization for all personnel is through improved training. 
Training that will help personnel benefit from the full potential of the improved technology 
instead of allowing these personnel to use the new technology simply to do things the same old 
way is needed before the benefits of improved technology can be fully realized (Moses, 2000). 
The framework ARI chose to organize their plan of attacking this problem consists of five topics 
(see Figure 1), each emphasizing a different area in need of research. 


Strategies for Training on Demand 


What training should be done how, when and where 
using soldier-centered distributed/digital 
environments? 


What assessment 
ensure that requi'r 
retained? 




tools are needed to 


;Kjjf)evels are attained and 


Determine Digital TsS®®gaining Requirements 

How should training requirements analyses change 
for digital tasks and for hybrid digital -conventional 
environments to determine distraining needed for 
producing and sustaining hj-gHlperformance soldiers? 


ents analyses change 


producing and sustaining high 

iK# 

\bM\y 


Certify Required Digital Skill Levels of Soldiers 


_ IMiik _ 

Train Adaptability 

What training methods work best to prepare 
^ jllsoldiers for frequent software/equipment 
jjjj ||uchanges combined with ad hoc and varied 
;; missions (including combat and special 
assignments)? 


Prepare Digitally-Linked Teams 

What methods work best to prepare leaders and to train 
widely dispersed soldiers linked as teams and units 
through digital networks? 


Figure 1. Five research topics for training the digital soldier (Moses, 2000) [emphasis added] 


As highlighted in Figure 1, ARI believes that training adaptability is a key aspect of training for 
successful battlefield digitization. Moses (2000) suggests that “Soldiers will have to go beyond 
minimal operation of digital workstations to be effective” and that training must shift its 
perspective from training in “what to think” to training in “how to think.” A key goal identified 
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in Moses (2000) for digital skills training is to “develop training methods and strategies that 
promote adaptive/flexible and innovative behavior.” The purpose of the Aptima/gOE two phase 
research and development is to meet this goal. 

A survey conducted on a class of soldiers training on the AFATDS system highlights the 
correctness of ARI’s belief that digital skills is an important arena for training at the dawn of this 
century. In general students reported a broad range of previous experience with computerized 
systems. As indicated in the figure below, some students reported no or very little experience 
with computers, and a larger percentage reported no experience with computer networks. This is 
significant because the kind of adaptable behavior that is desirable in recruits will come in part 
from substantial experience with computer technologies. When asked “What most helped them 
in their training,” many responded that previous experience with computers was key. Yet it is 
clear that this experience is by no means the norm among recruits. Detailed results from all 
questions are available in Appendix A: Results of AFATDS Training Student Survey 
Administered Week of 10.16.00 at Net Team Training, Fort Sill. Some highlights can be found 



Figure 2: Responses of a sample of AFATDS students to two survey questions 


The challenge of adaptive digital skills training 

The idea of “training adaptability in digital skills” is, on the surface, an appealing and intuitive 
one. To make it a reality, however, requires addressing fundamental and challenging issues in 
training design and training effectiveness measurement. The major challenges are: 

• Identifying a “core” set of digital skills that transfer across tools. 

While there are a multitude of organizations that focus on developing tool-specific training 
and documentation, there is little prior work on defining cross-tool digital skills. Defining 
this core skill set requires an open mind, experience in the various military domains in which 
the digital warrior operates, expertise in knowledge elicitation using techniques such as 
cognitive task analysis, and expertise in structuring knowledge and identifying skill 
commonalties across different environments and tasks. 
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• Developing training that teaches adaptable, flexible knowledge bases and strategies for 
exploiting them rather than quickly outdated knowledge and skills about specific tools. 

Most training packages focus on teaching specific skills to be used in specific contexts, for 
the very good reason that training developers have expertise and experience in designing and 
delivering this type of training. Developing instructional content that permits learners to re¬ 
use and re-apply it in new contexts is a significant, but achievable goal for training 
developers. Training individuals to adapt, change, and learn—as opposed to fit square-peg 
knowledge into a round-hole problem—in response to a changing environment is a much 
more difficult task than imparting specific knowledge. 

• Measuring the effectiveness of adaptability training in preparing learners to cope with 
new, unfamiliar environments and problems. 

Most measures of training effectiveness assess the ability of learners to perform those 
specific tasks for which they have been trained. Measuring transfer of training to new 
environments is much more challenging, and requires, for example, an underlying model of 
the relationships between latent constructs such as skills and performance in order to predict 
which skills should transfer to new tasks and to predict the expected measurable effects of 
training on new-task performance. 

We have begun to develop a digital-skills training package that is based on contemporary 
cognitive theory and training practice, provides today’s warfighter with a deep foundation of 
digital skills and knowledge, provides opportunities for practice in multiple contexts of use, 
teaches focused learning strategies to maximize the transfer of prior knowledge, and delivers 
training content directly to operational settings using modem distance-learning methods. Our 
approach is based on the fundamental premise that failures of digital-skills transfer are often 
failures of shallow, tool-specific knowledge to transfer to new contexts in which—despite 
similarities in the tools and operations involved—contextual or tool related changes make it 
difficult for learners to leverage what they already know to support new learning. There are 
several sources of evidence for this premise in the psychological literature (detailed below). In 
response to this, we propose that training be designed to provide a foundation of general digital- 
tool concepts and strategies for mapping these concepts to new applications will provide a 
deeper base of knowledge and cognitive learning strategies to insure rapid transfer. 

1.1 Phase I Objectives and Results 

The goal of Phase I was to show how a training program designed to increase adaptability in a 
subset of digital skills could be developed. Our team for Phase I—Aptima and gOE—began its 
Phase I effort with a strong background in both training and digital skill use. Aptima has 
expertise in digital skill use and training development; gOE has expertise in training 
development and training validation. 

Our overall objective in Phase I was to provide a proof of concept that shows the feasibility of 
developing an adaptive digital skills training package. Our challenge in Phase I was to create an 
example of a training program that is both general enough to train for adaptability in digital skills 
regardless of the context, yet specific enough that its effectiveness could be observable when 
used for a specific digital device. 
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To do this, we needed to focus on a specific Army training need and a specific type of digital 
device for which soldiers need to be trained. Using an example allows us to show how the 
device-specific tasks and actions that are currently being trained can be linked to underlying 
principles, knowledge, and generalizable digital skills. 

In Phase I, we made progress in addressing each of three major challenges identified above: 

• Identify core underlying digital skills for a selected domain. 

In Phase I, we focused our effort on US Army field artillery digitization and learned what 
digital tools are used, how they are updated, and how soldiers are currently taught about 
these tools. This domain was selected because it is currently a focus of ARI research on 
training for digitization and because one of the Aptima team members has extensive 
experience in this area. 

• Develop an innovative method to teach these skills in such a way that they can be transferred 
to other problems and situations. 

In Phase I, we conducted an extensive review of the literature on training for adaptability and 
developed an instructional design approach based on learning theory. The subgoal learning 
model provides a framework for designing instructional materials that help learners adapt old 
procedures to new problems. 

• Develop a plan for measuring the effectiveness of transfer of training. 

The final product of Phase I is a design for a validation study, coordinated with ongoing 
device-specific training for field artillery units, that will allow us to test the effectiveness of 
our training-for-adaptability approach. 

To accomplish these objectives, we had five technical goals in Phase I: 1) Gain an 
understanding of the capabilities and context of use of existing and soon-to-be-deployed 
digital tools, 2) Determine what fundamental knowledge is necessary to use these digital tools, 
3) Draw on human learning theory to determine the best approaches to maximize learning and 
knowledge transfer across domains, 4) Develop and demonstrate a proof of concept training 
program to facilitate human learning of digital tools, 5) Design a validation study to evaluate 
the training program. These goals resulted in a proof of concept for a training program and a 
plan for its validation. Following is a brief explanation of how these objectives were achieved. 

A more in-depth description can be found in sections 2-6. 

Phase I, Objective 1: Gain an understanding of the current and future digital tools 

Accomplishment: Identified US Army Field Artillery as the focus for Phase I, and learned what 
digital tools they use, how they are updated, how soldiers are taught about tools. _ 

The domain of Artillery was chosen as the focus of our Phase I effort for a number of reasons 
including a) on-going ARI research on training in this domain, b) the availability of in-house 
expertise in this area, and c) access to local experts. By taking advantage of a local Army 
Reserve Artillery Unit, we have gained access to experts trained in the use of two iterations of 
Artillery-support software: AFATDS and IFSAS. We were also able to visit SFC James 
O’Connor of the CECOM NET (the team that, until recently, was responsible for training 
delivered to soldiers receiving an updated version of AFATDS) and Mr. Roger Baker of 
Engineering and Professional Services (EPS; the organization currently responsible for AFATDS 
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update training), as well as other EPS personnel and AFATDS instructors. These opportunities 
have helped us develop an understanding of the domain, the ways that these tools get used to 
support the work done in the field, and the vision of the digital tools of the future. 

Phase I, Objective 2: Determine what fundamental knowledge is necessary to use these 
tools 

Accomplishment: We interviewed AFATDS trainers and EFSAS users, worked with our in- 
house expert to identify the digital skills and knowledge that underlie AFATDS and IFSAS use, 
and we observed AFATDS training delivered by EPS. 

Once we had a general understanding of the Field Artillery domain, we began to investigate 
more deeply the specific tasks performed by Field Artillery soldiers, focusing on those digital 
skills exercised when using the computer systems. To do this, we were fortunate enough to be 
able to sit in on some AFATDS training performed by EPS at Fort Sill, OK. During our 
observations we concluded that, although the training received by the soldiers comprehensively 
covers specific behaviors necessary to perform the requisite AFATDS operations, the soldiers 
did not receive training that covered 1) fundamental computer background important for 
understanding AFATDS components and operations, 2) troubleshooting and problem solving 
skills, and 3) how this AFATDS training really relates to the soldier’s job as a Field Artillery 
personnel (e.g., why is this training so important; how does the training map onto the soldier’s 
assigned tasks). Without the background and base-knowledge that these three areas afford, the 
trainee will have a difficult time comprehending and retaining the AFATDS training. After 
thoroughly reviewing the training literature, we felt that the soldiers would most benefit from 
training that addresses these areas. 

Phase I, Objective 3: Develop a sound training approach to maximize learning and 
knowledge transfer 

Accomplishment: Developed training-for-transfer approach based on the training and learning 
literature. _ 

Our goal was to develop a general training approach that is inspired by current theory and 
practice in the training and learning literatures, and is flexible enough for use in any context 
where learners use digital tools in service of their domain tasks. We went to this literature with 
the goal of developing an approach to training that helps learners to see relationships between the 
tools they use and the structure of the domain tasks which they presumably know well (although, 
we recognize the potential weakness of this assumption, and plan to build in reinforcement 
training that makes this structure explicit). Using the domain task structure as a learning 
scaffold, we additionally teach fundamental digital principles that help learners to understand, 
given the context of tool use, the kinds of tool interaction that are possible to get the job done, 
and thus the kinds of expectations they can have for software given the task at hand. This two¬ 
layered approach (that maps domain tasks to tools, and tools to digital implementation) has been 
implemented in Phase I for one digital skill found to be crucial for the AFATDS - building and 
operating an electronic network. 

Phase I, Objective 4: Deliver proof-of-concept training program. 

Accomplishment: We have developed a training program to illustrate our idea of linking domain 
specific knowledge to fundamental digital skills for computer networks. _ 
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At the conclusion of Phase I, we demonstrated how our theory-based approach can be applied to 
training for the fundamental digital skills that are associated with the selected AFATDS task, 
computer networks. The self-paced training program is written in HTML and can be delivered 
on a CD. 

Phase I, Objective 5: Design validation study 

Accomplishment: We designed a validation study to test the effectiveness of the training in 
promoting the ability of learners to transfer what they have learned to new problems. _ 

At the conclusion of Phase I we developed an Experiment Plan that details how we plan to 
evaluate both degree of learning and transfer success for the training package. Because our 
training on fundamental digital knowledge and skills is complementary to detailed training on 
the specifics of operating a digital device, we plan to closely coordinate our data collection in 
Phase II with AFATDS training delivery. 

1.2. Task Selection for Demonstration 

As stated earlier, we chose Field Artillery as the domain of focus for our Phase I effort. The 
rationale for this decision was based on on-going ARI research on training in this domain, the 
availability of in-house expertise in this area, and access to local experts. By taking advantage of 
ARI contacts at Fort Sill, OK, we were able to visit AFATDS trainers at Fort Sill, as well as sit 
in some of the training designed to introduce Field Artillery Soldiers to updated versions of the 
AFATDS system. Our in-house expert, a 15 year member of the National Guard, works in the 
Battalion and Battery levels of the Fire Direction Center. He has expertise in AFATDS’ 
precursor, IFSAS, and has been able to provide names and contact information for soldiers with 
both AFATDS and IFSAS experience. These opportunities have afforded us occasion to develop 
an understanding of the domain, the ways that these tools get used to support the work done in 
the field, and the vision of the digital tools of the future. 

One of the first conclusions we were able to draw from our observations and conversations is 
that the entire field of Artillery is quite massive and the AFATDS system is consequently rather 
complex, the capabilities of which are described in more detail below. A training program that 
will address the entire AFATDS system will be equally complex, so to successfully complete 
this first phase of the project we chose to focus our proof of concept training on just one of the 
many aspects of the AFATDS system, network communications. Our choice was guided by the 
consideration of the importance of the various system aspects, and how those aspects overlap 
with our in-house expertise. 

1.3. Document Overview 

The remainder of this document will elaborate on the introductory presentation of the Training 
Adaptability for Digital Skills effort. The following are brief descriptions of the remaining 
sections of this document. 

2. Understanding current and future required digital skills: Describes how we gathered the 
information to understand current and future digital skills, and outlines our findings. 

3. Determining fundamental knowledge necessary to use these tools: Contains an overview of 
the base knowledge that is needed to effectively use the digital tools. 
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4. Developing sound training approach : Illustrates how the training approach was developed. 

5. Developing proof of concept training program : Describes the proof-of-concept training 
program. 

6. Designing a validation approach : Presents an overview of the validation approach. 

7. Technical Objectives for TADS : Presents an overview of the plans for accomplishing Phase II. 

8. Plan for TADS : Describes the tasking schedule we will follow to implement Phase II 

9. Commercialization Plan : Contains our assessment of potential post Phase II TADS 
application and a plan to achieve these goals. 

2. Understanding current and future required digital skills 

The first task for the Aptima/gOE team was to gain a better understanding of current and future 
required digital skills. Ideally, this would be done for all domains requiring digital skill 
knowledge; however, to keep the task manageable we chose to focus on Field Artillery. The 
domain of Artillery was chosen for a number of reasons, including on-going ARI research on 
training in this domain, the availability of in-house expertise in this area, and access to local 
experts. By taking advantage of a local Army Reserve Artillery Unit, we have gained access to 
experts trained in the use of two iterations of Artillery-support software: AFATDS and IFSAS. 
We were also able to visit SFC James O’Connor of the CECOM NET Team (the team that, until 
recently, was responsible for all AFATDS update training) and Mr. Roger Baker of Engineering 
and Professional Services (EPS; the organization currently responsible for all AFATDS update 
training). These opportunities have allowed us to develop an understanding of the domain, the 
ways that these tools get used to support the work done in the field, and the vision of the digital 
tools of the future. 

To familiarize ourselves with the chosen domain (i.e., Field Artillery), we tapped into several 
different sources 1 and performed a high-level task analysis. From this initial task analysis it was 
established that the United States Army Field Artillery is a combat arms branch whose units 
engage enemy forces with cannon, rocket, and missile fires and acquire targets for combined 
arms operations to accomplish the mission. The Field Artillery team is divided into four major 
components: 1) the Fire Support Team (FIST) that works within each company to develop fire 
support plans that protect the company; 2) The Target Acquisition team that uses radar to acquire 
deep targets; 3) the Firing Battery who deliver the right type of artillery fire at the right time and 
place on the battlefield; and 4) The Fire Direction Center (FDC) who translates the FIST’s call 
for artillery fire into information used by the Firing Battery. To accomplish their part of the 
overall mission, the field artillery units use advanced computer and communications systems to 
do a number of things including produce calls for fires, process calls for fire, and plan the fire 
mission - and it is this reliance on advanced technology to accomplish their part of the mission 
that made the Field Artillery team ideal for the purposes of the current project. The digital skills 
we have chosen to focus on are those necessary to operate the IFSAS and the AFATDS, two 
versions of a computerized support system that allow Field Artillery leaders to gather 


1 Much of this information was acquired through personal correspondences with Field Artillery personnel, a visit to 
CECOM NET Team and EPS personnel at Fort Sill, observations of AFATDS update training at Fort Sill in 
October, 2000, and from Fort Sill and other relevant internet sites. 
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information, pass orders and compute data. IFSAS is an older system still in use by certain 
facets of Field Artillery, whereas AFATDS - a newer system that is continually being upgraded 
- is used more globally in the Field Artillery because of its increased versatility. During the 
interviews and observations we learned about the problems experienced by soldiers because of 
the frequent changes in AFATDS, including both problems with remembering the old version 
and problems with learning the new version. Digital skills training will be applicable to these 
systems as they change in the future, as well as to any other military and non-military computer- 
based systems. 

AFATDS is a totally integrated fire support Command and Control (C2) system that processes 
fire mission and other related information to coordinate and optimize the use of all fire support 
assets, including mortars, field artillery, cannon, missile, attack helicopters, air support, and 
naval gunfire (e.g., see Boutelle and Filak, 1996). AFATDS supports the five Fire Support 
functional areas: 

• Fire Support Planning; providing integration of field artillery, mortars, naval gunfire, 
and air support into the force commander’s scheme of maneuver. AFATDS helps create a 
Fire Support annex to the commander's Operation Plan (OPLAN) and a Field Artillery 
Support Plan. 

• Fire Support Execution; guided by fire support and field artillery support plans, it 
performs sensor employment, target processing, attack systems analysis, and target 
damage assessment. 

• Movement Control; managing and coordinating the movement of field artillery units 
and sensors and coordinating the movement of fire support units and sensors. 

• Field Artillery Mission Support; including functions logistically supporting the field 
artillery system, it creates and maintains supply inventory files, supply requests, and 
supply reports. 

• Field Artillery Fire Direction Operations; including the collection and maintenance of 
weapon, fire unit, and ammunition status data required for day-to-day operations. This 
information is provided in either detailed or aggregate form to appropriate Operations 
Centers in support of both planning and execution requirements. 

As AFATDS is upgraded over the next decade, the plan is not for substantial change but rather 
for improvements that build on the system as it is today. Specifically, the plan is for it to become 
more automated, have more software modules incorporated into it (to enhance AFATDS joint 
capabilities), and ultimately become fully interoperable with the automated systems of all 
services. 

3. Determining what fundamental knowledge is necessary to use 
these tools 

Once we had a general understanding of the Field Artillery domain, we began to investigate 
more deeply the specific tasks performed by Field Artillery soldiers, focusing on those digital 
skills exercised when using the computer systems. For example, some of the tasks performed by 
Field Artillery units while using the computer support systems are as follows: fire support 
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planning and coordination, fire support execution. Field Artillery movement control, and Field 
Artillery mission support. Each of these tasks can be broken down into smaller (i.e., more 
specific) tasks; for example, fire support planning and coordination consists of a number of 
smaller tasks including the following: plan target list, edit target list, pre-coordinate targets, 
create a group of targets, and create a series of targets. These smaller tasks can, in turn, be 
divided into even more specific subtasks that are unique to the particular computer system being 
used, one example being the task “create a group of targets” - in IFSAS the operator first selects 
“FIRE PLANNING” from the main menu, then selects “INST,” and then goes through a series of 
selections to specify the fire plan name, the specific targets, a group name, a phase, etc.; whereas 
in AFATDS the operator first selects “Targets/Groups/New...” and a new window opens, the 
operator then creates the new group, and clicks “OK.” For our purposes, we are focusing on the 
top and middle levels of tasks - those that decouple the operator’s task from the specific 
computer system - because we want our training to be useful regardless of the details of the 
computer system being used 2 . 

Basic digital skills and knowledge used by the IFSAS and AFATDS users include networking 
knowledge (e.g., host name, IP address, SCSI,), communication and planning skills, geometry 
computation skills, data analysis, data base management, visualization/mapping, software 
installation, and trouble shooting skills. 

The general digital knowledge and skills that have been identified in Phase I include: 


Knowledge of 
computer networks 

Knowledge of 
computer components 

Knowledge of 
computer operating 
systems 


Understanding of 

electronic 

communications 

Computer experience 

General intelligence 

General mechanical 
efficacy 


Computer efficacy 
Planning 
Decision making 
Geometry 
Mapping skills 


Although the soldiers are comprehensively trained in many of these areas in AFATDS training, 
three areas the training does not cover thoroughly are 1) fundamental computer background 
important for understanding AFATDS components and operations, 2) troubleshooting and 
problem solving skills, and 3) how this AFATDS training really relates to the soldier’s job as a 
Field Artillery personnel (e.g., why is this training so important; how does the training map onto 
the soldier’s assigned tasks). Without the background and base-knowledge that these three areas 
afford, the trainee will have a difficult time comprehending and retaining the AFATDS training. 
After thoroughly reviewing the training literature, we felt that the soldiers would most benefit 
from training that addresses these areas. 


Integration with AFATDS Training 

Figure 3 represents the current training need for AFATDS specialists. Currently, the soldiers 
receive extensive training in their Military Occupational Specialty (MOS) and in the tools that 
they will use in performing the duties required of their MOS. For example, for the new MOS 


2 It is important to note that current computer training generally focuses on the lowest level (i.e., computer system 
specific) subtasks. 
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13D (the 
AFATDS 
specialist), the 
soldiers receive 
extensive 
training 
regarding the 
specific tasks 
expected of 
them (see 
Table 1). Once 
they have had 
sufficient time 
to learn what 



Figure 3. Current training of the US Army Field Artillery AFATDS specialist 


tasks are 

expected of them, they are taught how to use the computer system (the AFATDS) designed to let 
them do their tasks. During AFATDS training, the soldiers are thoroughly trained in the specific 
actions necessary to successfully operate the system. 


Table 1. Sample Tasks from the MOS13D Task List 


TASK NUMBER _ TASK TITLE _ 

061-275-2001 Employ Field Artillery communications systems 

061-275-8010 Use field wire laying techniques 

061-279-5017 Maintain operational graphs on a map 

061 -280-5005 Perform manual safety procedures 

061-299-5005 Process meteorological data 

061-299-5017 Decide to attack a target 

061 -299-5019 Direct the massing of artillery fires 

061-299-5109 Prepare a situation map 

061 -300-0006 Develop routing for target indicators 

061-300-0007 Develop automatic purging for target indicators and suspect targets 
061-300-0008 Manually input suspect targets and target indicators. 

061-300-0025 Load initial vector map to AFATDS workstation 
061-300-5002 Establish TAFCS communication configurations (AFATDS units) 
061-300-5003 Incorporate the printer into the TAFCS (AFATDS units) 
061-300-5006 Process geometry data in the TAFCS (AFATDS units) 

061-300-5007 Up data unit data in TAFCS (AFATDS units) 

061-300-5008 Enter the field artillery support plan text (AFATDS units) 
061-300-5009 Perform trouble shooting procedures on the TAFCS (AFATDS units) 
061-300-5010 Configure received message types (AFATDS units) 

061-300-5011 Process message for transmission (AFATDS units) 

061-300-5012 Disseminate information via data distribution (AFATDS units) 
061-300-5015 Process target information (AFATDS units) 

061-300-P001 Initialize the AFATDS 

061 -300-P002 Shutdown the AFATDS workstation operations_ 
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Currently, these two sessions are taught as being somewhat unrelated so the soldiers often 
experience great difficulty when trying to learn the AFATDS system, and they find it extremely 
difficult to learn the new system as the AFATDS progresses. We think our training can help 
soldiers adapt more easily to the computer system designed (e.g., the AFATDS for MOS 13D 
soldiers) to let them do their job as well as prepare the soldiers to learn the new system more 
easily as AFATDS updates. 

A graphic representation of where our training program will fit into the education currently 
experienced by the Field Artillery soldiers can be found in Figure 4. The training that we 
propose to develop will bridge the gap between these two aspects of the curriculum to make the 
AFATDS training—both current and future—more effective. Our training will help remedy 
these problems by forming a bridge between the specific MOS knowledge and the detailed 
AFATDS instructions by training the soldiers to focus on bigger picture knowledge associated 
with tasks. We will help soldiers map job tasks and subtasks that they leam comprise their MOS 
onto specific actions they are trained to perform with the computer software. 



Figure 4. Relationship between current and proposed training 


Following are some real-life experiences illustrating not only that soldiers, in this case Personnel 
from the National Guard, have a difficult time learning the necessary digital skills but also that 
the ramifications of this faulty learning can be quite detrimental. The following story is from 
Mr. Paul Titus; a member of the Aptima team who is also a member of the National Guard. 


DAS WO 1-00-C-3014 

Training Adaptability in Digital Skills 


12 


AP-P-1128 




Aptima, Inc. 


Real-World Problems 

All of the computers in my battalion were upgraded to version 11 of the IFSAS (battalion level) 
and BCS (firing battery level) software. The major differences between the old and the new 
systems were in the communications area, which seems to be the toughest part of the system 
for the soldiers to learn. The change was made to allow unit's using IFSAS to communicate 
with units using AFATDS by allowing the operator to select the communications platform that 
brings IFSAS into the AFATDS communication world. 

There was a four day training class covering the differences between version 10 and version 11 
and the soldiers who attended the class got a 3-ring binder that contained a document referred 
to as a job aid that provides step-by-step instructions about how to initialize the communication 
settings. One of the differences between the old and new versions of the IFSAS software is that 
the new system now includes a Unit Reference Number (URN: an arbitrary 8-digit number) as 
an additional field in the form. The students were told that in the AFATDS system, this field 
would be for an IP address and each computer in the network must have a URN as well as keep 
a list of the other computers' URN. To this point, everyone seemed to understand that it was 
important to have the URN, but had no idea of what it is used for. Since I didn't attend the 
training class, I assumed that the URN would be used at a later time during the initialization 
procedure or when we were actually trying to communicate with another computer. Needless to 
say, I was wrong. We never used that URN again, and a number of the soldier's were upset 
that they had to waste time entering these 8-digit numbers for every computer, then not use 
these numbers. It got to the point where some of the computer operators didn't pay close 
enough attention when entering these numbers, resulting in communication failures. It turns out 
that the numbers are used in the headers of all transmissions and are checked against the list of 
subscribers that your computer is supposed to communicate with. This was not explained 
during the training and once I explained what it was used for, the operators then understood the 
importance of accurately typing those numbers. Because the training material did not explain 
the importance of these numbers, other than the statement that each computer must have the 
same numbers for all the computers in the network, our unit lost 3 hours of training. 

After solving that problem, we still could not establish communications between computers, so 
we began more troubleshooting. It turns out that the computer operators were entering the 
values used in the job aids for each field. The training material assumed that the computers 
would be using SINCGARS to communicate, and that's what our unit uses as well so everyone 
assumed that by using those values we would not have any problems. We were provided a 
communications troubleshooting matrix, so when we kept experiencing the same error condition 
I scanned the matrix but couldn't find the error message. Our only recourse was to troubleshoot 
by changing the values in each field one at a time and restarting the systems. After losing 
another 5 hours of training time, we came across the correct settings to communicate. The 
problem was when an operator chooses SINCGARS as the radio device, the default for the 
modulation field is NRZ. That was also the setting from the training material. Once we got 
around to changing that field from the default to 1200/2400 FSK, we were no longer getting the 
error message and were able to send messages between computers on our radio network. I 
spent another 2 hours once we shutdown training looking through the job aid for any 
explanation of the modulation field to no avail. We all agreed that since NRZ is the default value 
there must be a wrong value entered in another field, but since we established communications 
and had lost an entire training day we decided to use these settings 
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We are not suggesting that our training will solve all the problems and roadblocks experienced 
by computer users; however, it could give computer users more knowledge and understanding 
that can be used as tools with which they can attempt to solve problems as they emerge. Mr. 
Titus is well trained in computers, so if he had not been around to point out the importance of IP 
addresses the soldiers may never have realized the cause of their problems. Our training will 
introduce novice computer users to the topics that are most important and thus the first to 
consider when experiencing difficulties. 

4. Developing a sound training approach to maximize learning and 
knowledge transfer 

Our ultimate goal for this two phase SBIR program is to develop a general training approach 
inspired by current theory and practice in the training and learning literatures that is flexible 
enough for use in any context where learners use digital tools in service of their domain tasks. 

We went to the learning literature with the goal of developing an approach to training that helps 
learners see relationships between the tools they use and the structure of the domain tasks which 
they presumably know well (although, we recognize the potential weakness of this assumption, 
and plan to build in reinforcement training that makes this structure explicit). Using the domain 
task structure as a learning scaffold, we teach fundamental digital principles that help learners 
understand, given the context of tool use, the kinds of tool interaction that are possible to get the 
job done, and thus the kinds of expectations they can have for software given the task at hand. 

To do this we use a “two-layered” approach that maps domain tasks to tools, and tools to digital 
implementation, a prototype of which was implemented in Phase I for one digital skill found to 
be crucial for the AFATDS - building and operating an electronic network. 

Theory-Based Approach to Training for Adaptability 

During Phase I, our investigations of the army’s artillery fire support system revealed several 
issues that the proposed training package must address to be successful. First, a system like 
AFATDS is complex, and for a novice in computer systems, its interface is un-intuitive. Experts 
in AFATDS note that not only must trainees understand tactical operations very well to use 
AFATDS, but they must have a very good mental model of the system and its functionality in 
order to trouble-shoot in the system. For example, there are a number of functions for which the 
user cannot check the accuracy of recently inputted data and must test a long list of possible 
trouble-spots to find the cause of error. Second, the current training targets a user who is perhaps 
already somewhat knowledgeable about computer systems and understands the inner workings 
of the internet. For example, to set up communications within AFATDS, the trainee must 
understand the components of an electronic network, which is far more than simply typing in 
every host’s address correctly. Even when all members of a network-in-the-making know each 
other’s host addresses, the network may fail because of a long list of other requirements, any one 
of which may not have been met. 

In a training situation such as this, with complex material to teach quickly to novices in computer 
systems, it is very difficult to avoid teaching “buttonology” and lists of procedures. That is, one 
eventually must train specific steps and procedures. Unfortunately, procedures learned in long 
lists inevitably become a kind of “inert” or un-usable knowledge if the learner lacks a conceptual 
framework for mentally storing, organizing, and accessing that much complex information. The 
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proposed training package delivers a conceptual framework plus a way of guiding students to 
apply that framework in learning the specific buttonology of a particular system like AFATDS. 
The package serves as a “prelude” or a bridge to procedural training in a specific system to make 
it the most effective it can be. 

Teaching "Expert Models ” of Complex Conceptual Knowledge 

The training solution for teaching complex and unintuitive material to novices is instruction in 
the expert mental models for organizing that material. There are two types of mental models that 
individuals need for learning to operate complex devices. One type of model is known as a 
device model (Kieras & Bovair; 1984; Folz, Davies, Poison, & Kieras, 1988); for example, the 
expert’s mental model of the electronic device known as an internet. Particularly with a complex 
system such as a network, the trainee might have a mental picture of a diagram of such a device 
showing the major components and the information flows in a network. With basic conceptual 
knowledge of what makes a network, database, visualization tool, or data analysis package 
work—for example, its components, resources, backup capabilities, power and information 
flows—the AFATDS trainee could trouble-shoot in a system and apply his or her knowledge to 
learning a new system or software version. A low-level example is that with a good device 
model, trainees “know that” computer systems have backup power capabilities (e.g., an 
Uninterruptible Power Supply). This factual, conceptual knowledge supports the trainee in 
understanding not only such basic features as where the system gets power, but the implications 
of shutting down or turning off the system. The trainee who believes data is always lost when the 
system is not properly shut down is a trainee with brittle, inflexible, and even somewhat 
superstitious knowledge. She or he is willing to use only one method—the “recommended” 
one—for shutting down the computer, when in fact, different methods (turning the system off, 
cutting the power supply) exist for different situations (e.g., protect data by destroying it when 
under attack). The trainee’s belief is rational, but based on an erroneous, possibly nonexistent, 
mental model of the device. 

Training design . In the instructional module of our training package, students are 
presented with hyperlinked lessons on a device model. Students see diagrams 
representing a comprehensive but also generalizable device (e.g., for a generalizable 
network). Text describes the components and functions; interactive highlighting 
(balloons-style) further illuminate important issues and topics. Structure in the lessons 
guide students to navigate the pages of content from overviews of the device to more 
detailed information about sub-components and their inner workings. Special targets 
of instruction are terminology, the functions of components, the links between 
components, and communication tasks (e.g., building a network, sending data, 
receiving messages) that are served by different sections of the device. 

A device model is only part of the knowledge needed to support expert performance with a 
complicated device. This is because the steps and procedures for interacting with the device may 
also be complex and counter-intuitive, especially when the device is a computer system with a 
fairly un-instructive interface. Thus, the second mental model that the expert has is a procedural 
model, or a set of goal structures for the practical “how to” knowledge of interacting with a 
system (Catrambone & Holyoak, 1990; Catrambone, 1998). For example, the expert user knows 
that there are steps associated with the goal “shut down the system.” Some of the steps are 
further organized into sub-goals such as “shut down in an attack” or “shut down so as not to lose 
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data.” In an attack, the expert executes steps that turn off the system the most quickly and keep 
data from falling into enemy hands. In another situation, however, the expert executes a different 
set of steps for carefully preserving data while shutting down. Each sub-goal is a mental category 
for remembering a group of unintuitive steps or procedures such as arcane keystroke 
combinations and navigations through different menus together in a meaningful way. When a 
trainee is presented with procedural instruction that explicitly groups and labels steps according 
to the relevant goals and subgoals, she or he can not only learn a system well, but apply that 
learning to new versions of software (Novick, 1988; Catrambone, Experiment 1,1998). Having 
these mental groupings for steps, a learner will be better prepared to remember information about 
steps. Additionally, when faced with a new version of software to leam, the learner will 
remember the various “shut-down” goals from the previous system and look for steps and 
procedures in the new system that accomplish those goals. Aided by this well-organized 
knowledge of goals and steps, learners may even analogize about individual steps from the old 
system to the new one. For example, if in the old system the learner had to use menu options to 
shut down the system without losing data, then in the new system the learner would analogize 
that group of menu steps to the new system, searching for and trying out menu-driven shut-down 
procedures. 

Training design . Currently in the instructional module of our proposed training 
package, students are presented with brief lessons on a procedural model. Students 
see diagrams representing the generalizable device in question, as well as diagrams 
representing a process they are more familiar with (e.g., the USPS) and similarities 
between the procedures are pointed out. The availability of the device diagrams will 
help students integrate information from the device model with the procedural 
model. 

Guiding Application of Knowledge 

With knowledge of a type of device (e.g., an electronic network) and the general procedural 
goals for interacting with it (e.g., steps for building a network), learners must be shown how this 
critical information applies to a specific system. While device and procedural models are 
powerful concepts that guide the trainee to leam new systems, they are also general models that 
must be adapted to the specific buttonology of a system. Some guidance must be given initially 
in applying those concepts as this kind of learning-transfer activity is traditionally a stumbling 
block for learners and training packages alike (for a review of this issue, see Singley & 

Anderson, 1989). Our solution to this issue is to guide students through a module that acts as a 
Skills Bridge between the conceptual instruction in device and procedural models and the 
systems-operator instruction (e.g., for AFATDS) currently delivered by the Army. Practice in 
such a module results in more refined understanding of the device and procedural models and in 
mental rules for analogizing one set of procedures to a goal in a new system. Just because 
learners are asked to draw on old knowledge to solve a similar but new problem does not mean 
that the learner will know what to do first in that situation. Practice in the Skills Bridge module 
will give students practice using analogy heuristics such as thinking about the general goals of a 
device first, putting off thinking about superficial features like key-stroke combinations, and 
modifying an old procedure several ways rather than trying to use it identically in the new 
system. 
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Training design . After they have received training in a generalizable device model 
and a procedure model, students are presented with lessons in a Skills Bridge that 
links what they have learned in the TADS training to the specific system they are 
preparing to learn about (in our prototype - the AFATDS system). This Skills 
Bridge will assist the student in building a mental model, or schema (e.g., see Wyer 
and Srull, 1989), of the AFATDS system so that future association with that system, 
or similar systems, can be more efficient and productive. To create the Skills Bridge 
we used material supplied by AFATDS trainers as well as material in the AFATDS 
manual and leveraged the structure of the generalizable device and procedural models 
as well as principles of instructional design (e.g., how to sequence instruction; how to 
present information effectively; how and when to evaluate). The Skills Bridge will 
help the student understand how the general training will be useful in the system 
training and thus increase motivation; it will also increase the probability of training 
transfer by using the terminology and illustrating the basic device and procedural 
models that students have learned so far in the new context of a new system. 

Gauging Students ’ Progress Continually 

In a training package for teaching complicated cognitive skills, one must ensure that all learners 
are keeping up with the instruction. The device model, procedural model, and Skills Bridge are 
modules of instruction that build, module by module, an integrated body of knowledge. Regular 
assessment will be necessary to assure that students can remember, for example, the components 
of an internet device and the procedural goals and sub-goals that must be met in building a 
network, before students attempt to apply this knowledge in a Skills Bridge module. 

Regular testing is also an opportunity to select the next level of informational complexity and 
instructional content to present to students. For example, early exercises in a module would give 
students practice in retrieving overview information about the general functions of a device. 
Later, when students show mastery of the earlier material, exercises would lead students to refine 
that knowledge further by solving “de-bugging” problems and testing hypotheses about the 
location of errors and the best ways to resolve them. The general plan of instruction flow from 
module to module also implements an instructional design from the simple to the more complex. 
For instance, students leam overview information about general devices before attempting to 
apply the knowledge to procedural goals. Once students have mastered general device and 
procedural models, they refine and deepen their knowledge by applying it in limited examples 
during the Skills Bridge module. After students have completed the proposed modules, they will 
be ready to again deepen their knowledge of devices and leam the complex workings of a 
specific system like AFATDS. 

To get the full benefit of the regular assessment, the student must receive timely and constructive 
feedback (Latham, 1989) for both correct and incorrect answers. Without such feedback, 
learning is unlikely to take place and the students will become unmotivated and frustrated. 

Training design . Each student begins the training with assessment questions to 
gauge their mastery level of the concept being trained. Assessment for each topic 
begins with simple, or basic, questions that gradually progress to more difficult, or 
complex issues. Each time they answer a question, they receive immediate feedback 
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about the correctness of their answer. If they answer it correctly they go on to the 
next harder assessment question; if they answer it incorrectly, they go to instruction 
focused on the specific construct of interest. 

Summary of Adaptability Training Approach 

Figure 5 summarizes the components of our approach, as discussed above. We will train 
learners to develop a general device model (e.g., a model of the components of a network) and a 
general procedures model (e.g., how one sets up a network). Then we provide a “Skills Bridge” 
that links this general knowledge to the specific training that soldiers receive on the AFATDS 
system. 


Digital Skills Training for Adaptability 


General 
Device Model 


General 

Procedure Model 



MOS 13D Job 

-Tasks 

-Subtasks 



Our training will 
form a bridge 
that connects the 
soldier’s varied 
training 



Specific 

Actions/ 

Buttonology 


Figure 5. Components of Proposed Training Approach 


Both the general device model training and the general procedure model training take place 
through progressively more difficult assessment and focused instruction for each topic (e.g., 
computer components, computer networks, electronic communications). Feedback is given after 
each assessment question is answered, and the Skills Bridges occur after each specific 
knowledge or skill (e.g., understanding Wide Area Networks) has been successfully mastered 
(i.e., the student correctly answered all questions for that training segment). Figure 6 depicts the 
flow of the TADS training program. 
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Figure 6: Flow of the TADS Training Program 


5. Deliver proof of concept training program. 

At the conclusion of Phase I, we have demonstrated how our theory-based approach can be 
applied to training for the fundamental digital skills that are associated with a selected AFATDS 
task - computer networks. The proof-of-concept training is written in HTML and uses 
JavaScript, so the students could theoretically take this training over the internet, or the training 
can be used from a CD, or it can be copied onto the hard drive of any computer so students can 
receive the training at their convenience. Figure 7 portrays the flow of this section of the TADS 
training program. Following is a description of the training program developed for the Phase I 
project. 
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Figure 7: Flow of the Computer Networks Section of the Training Program 


The training program begins with an introduction to the entire training program (See figure 8). 

In this general introduction, the training program is described and the student is told what they 
should expect as they continue with the program. When the entire training program is completed 
at the end of the second phase, the student will be able to choose any of the topics in the outline 
on the left to get further training; however, in this initial prototype, only computer networks 
training is available. 
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Figure 8. General Introduction page 


Because it is assumed that this training will be delivered with minimal instructor contact, the 
initial instruction is quite detailed. When a student clicks on the “continue” button at the bottom 
of the initial instruction page, they will see another instruction page that introduces the three 
types of pages the student will encounter throughout the training program (see Figure 9). 


DASW01-00-C-3014 

Training Adaptability in Digital Skills 


21 


AP-P-1128 







































Aptima, Inc 





ow me rrogram 


You vvi 2 cjspericncs three different pages faring this training. Click on one below to get more detailed 
iriormahon about it Or, if you prefer, you can bypass the introduction and simply htm the fraamn& 

mmm. 





tsrwssrrj 


x f-siiKtt »*!**► '*Umu i 5 - .4 •»<«.** ** 


* ifV *»"*>*■* * (»!#**• WlMOr* , "*¥*‘'*® 1 •}> 


Get more information about die Assessment Pages Get more information about the Instruction. 




Get more.infoTTnAjtion Ab^#, the 




***W-OW rV: I 


Figure 9. Introduction to the Three Types of Instructional Pages. 


By clicking on the specific page, a student can get a more detailed introduction of the 
Assessment, Instructional, or Task Mapping (i.e., the Skills Bridge) pages where each part of the 
page is explained in detail. Figure 10 presents the three introductory pages for the instruction 
pages. 


DAS WO 1-00-C-3014 

Training Adaptability in Digital Skills 


22 


AP-P-1128 


















Aptima, Inc 


AFATDS Prep Training>computer networks>general domain knowledge* instruction basic LAN concept 


Figure depicting 
concept being 
covered by the 
^jjjfelirograin 




Outline showing 
^ ^ ye you are 

Pull down menu so you can select 
a figure depicting a concept 
? similar to the one being trained to 
inul^the,trainJh^ program easier 


A Local Area Network of computers 
formed simply by direct cable (or wtr 
at least two computers A LAN can 1 
or a? large as hundreds The import ai 
computer on a given LAN has a due* 
LAN and its hardware - no routmg o 


Thorough 
description 
ofconcept 
and devices 


{How is this simitar to 



it screen 


Outline showing 


AFATDS Prep Training>computer networks>general domain knowledge>instruction basic LAN concept 

f^| Figure depicting 

Mm & concept being 

j|f||pr |gm^4,bythe 

Li.. 


Here you can see 
how a LAN is 
similar to the US 
postal service, the 
phone, or the radio 




. How is this similar to. 
j How is this similar to... 


Second! 



A Local Area Network of computers, or network nodes are 
formed simply by direct table (or wireless) connections between 


Outline showing 


AFATDS Prep Training>computer networks>general domain knowledge>instruction basic LAN concept 


Figure depicting 
concept being »[ 

^pwredby the 

program 



This is the figure 
depicting 

sinudarities between 




'-fi 


0, 


A Local Area Network of computers, or network nodes are 
formed simply by duett cable (or welts?) connections between 
at least two computers A LAN can be as small as two computers, 
or as large as hundreds The important concept is that every 
computer on a given LAN has a dareet connection to that specific 
LAlLandits hardware - no routing *r indirect conneebons apply 


'Thorough 
description 
ofconcept 
and devices 


Clicking on this bar will send you 
directly to the relevant assessment page 


Metaphor 

A local or regional office of the postal service ts like a LAN 
Homes and offices m the area all have direct connecbons to the 
local office through mad earners Mail going from one place within 
the local office's domain to another need never go to another post 
office. Aw 


concept and devices * 
being used to make the 
training program easier 
to understand 



iird screen 


Figure 10. Sequence of 3 screens describing the instruction page 
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Both the assessment and instructional pages have three introductory pages to illustrate the pull¬ 
down menu for the metaphor. In discussions with AFATDS instructors and students at Fort Sill, 
it was decided that no one metaphor would be perfect for all students, because of different 
backgrounds and different ways of thinking about the topic, so students should have a choice of 
metaphors from which to choose. To illustrate how to choose a metaphor from the pull-down 
menu, the introductory pages walks the student through selecting the USPS metaphor. 


When the student has completed the introductory pages, he can then begin the training program. 
The first screen the student will then see for the computer network module is the introduction to 
the computer network module (See Figure 11). Like the assessment and instruction pages, the 
student can choose from a list of metaphors included to help them better understand the topic. If 
they click on “Continue”, they are sent to the first assessment question (see Figure 12); if they 
click on “To Instructional Information”, they go to the first instructional page (See figure 13). 



In this lesson you will learn about computer networks. Any time you want to connect 
multiple computers, you need to do so on a network. AFATDS is a system that must be If, 

used with a network of computers. Knowing the basics of computer networks will not only 
help you with APATDS, but it will help you any time in your future when you need to *» 

connect multiple computers. Networking computers can be compared to the United States 
Postal Service, phone systems, and the radio - analogies used throughout this training. if 




Figure 11. Introduction to Computer Network Module 
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As the student navigates through the assessment and instructional pages, the training program is 
designed to give instantaneous feedback for the student’s answers to the assessment questions. If 
an answer is incorrect, a dialog box pops up stating that the answer was incorrect and the student 
is being sent to the appropriate instructional page (see Figure 14). The student must click on the 
“OK” button to make the dialog box disappear, and the student is then sent directly to the 
instruction page. 


'2tittp://n2.aptima.com^tflusfcfl8rt:alskiil«/asse^sni#nt%ompnet02.iitiiiJ - Microsoft Inte rnet Explorer ___ EBB! 



Figure 14. Students get immediate feedback for incorrect answers and are sent to the instruction pages. 


If the student answers correctly, another dialog box pops up explaining that they are correct and 
can either continue with the assessment questions (See Figure 15) or go on to the Task Mapping 
screen (see Figure 16). 
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For our Phase I and option product, we have developed a proof-of-concept training program for 
computer networks. For Phase II we will continue to develop and expand the training program 
to include all topics necessary for a solid base knowledge of digital skills. 

6. Design validation study 

Following is an Experiment Plan that details how we plan to evaluate both degree of learning and 
transfer success for the training package. Because our training on fundamental digital 
knowledge and skills is complementary to detailed training on the specifics of operating a digital 
device, we plan to closely coordinate our data collection in Phase II with AFATDS training 
delivery. 

Experimental Plan 

To deliver a valid training program at the end of Phase II, we have determined that a two-stage 
validation plan is called for. For the first stage, we will conduct an initial usability test and 
content validation study on the training as it was developed in Phase I. After further developing 
the training program, we will conduct a full validation study to ensure that the entire training 
program is valid. Both of these studies is explained in more detail below. 

Initial Validation 

For the initial validation we want to make sure the training we developed in Phase I is 
appropriate before expanding it to the full training package. This initial validation study will 
have 2 parts a) Conduct initial usability testing, and b) Conduct initial content validation study. 

A. Usability testing . The goal of usability testing is to determine how users who have not 
previously encountered the TADS system react to it. Specifically, what problems or 
difficulties do naive users encounter in proceeding through the training? What 
recommendations for improving the system do such users have? 

Typically, usability of a web site or web application is conducted in several different 
ways. These include “cognitive walkthrough” (e.g., Spencer, 2000), “heuristic 
evaluation,” (e.g., Neilsen, 1994), “thinking aloud protocol” (e.g., Rubin, 1994), focus 
groups, questionnaires, and feature and consistency evaluation. We propose usability 
testing of three types: cognitive walkthrough, thinking aloud protocol, and heuristic 
evaluation. Usability testing will be conducted on both the prototype and during the 
creation of the actual Phase II deliverable. Each of the planned usability evaluations are 
discussed briefly below. 

Heuristic evaluation (Molich & Neilsen, 1990; Neilsen, 1994) consists of designing and 
reviewing a system in order to assure that certain fundamental usability good practices 
are followed. That is, various criteria for success in this area are stipulated and examined 
in regard to a particular project. After Molich & Neilsen (1990 - note that some of the 
wording in the numbered points directly below is theirs), we list below some of the 
usability heuristics to be used in the design and review of the TADS system. 

1. Visibility of system status - The system should always keep users informed about 

what is going on, through appropriate feedback within reasonable time. 
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2. Match between system and the real world - The system should speak the users' 
language, with words, phrases and concepts familiar to the user, rather than system- 
oriented terms. Follow real-world conventions, making information appear in a 
natural and logical order. 

3. User control and freedom - Users often choose system functions by mistake and 
will need a clearly marked "emergency exit" to leave the unwanted state without 
having to go through an extended dialogue. Support undo and redo. 

4. Consistency and standards - Users should not have to wonder whether different 
words, situations, or actions mean the same thing. Follow platform conventions. 

5. Error prevention - Even better than good error messages is a careful design 
which prevents a problem from occurring in the first place. 

6. Recognition rather than recall - Make objects, actions, and options visible. The 
user should not have to remember information from one part of the dialogue to 
another. Instructions for use of the system should be visible or easily retrievable 
whenever appropriate. For the TADS system, this has implications on limiting the 
amount of screen scrolling that is required. 

7. Aesthetic and minimalist design - Dialogues should not contain information 
which is irrelevant or rarely needed. Every extra unit of information in a dialogue 
competes with the relevant units of information and diminishes their relative 
visibility. 

8. Help users recognize, diagnose, and recover from errors — Error messages 
should be expressed in plain language (no codes), precisely indicate the problem, and 
constructively suggest a solution. 

9. Help and documentation - Any help should be easy to search, focused on the 
user's task, list concrete steps to be carried out, and not be too large. It is anticipated 
that the TADS system, after a brief orientation, will have minimal need for additional 
help documentation. Nonetheless, where required as determined by good design and 
results of other usability testing, help will be made available. 

A second kind of usability testing that will be used in the TADS development is what can 
be called expert cognitive walkthrough (ECW, Spencer, 2000). Best for the earliest stages 
of product development, the ECW consists of designers “storyboarding” a process in the 
sense of anticipating the sequence and nature of the actions required of a user, and 
identifying any problems that the user might encounter in accomplishing the goals of the 
system. 

A third kind of usability testing is the so-called “thinking aloud” method (e.g., Rubin, 
1994), which is a kind of observational technique. In this method, users are asked to 
complete a task (in this case, to proceed through the TADS system), and further asked to 
voice their thoughts as they complete it. This method is ideal for identification of any 
aspects of a system that a user finds confusing, ambiguous in purpose, or difficult in 
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execution. In addition, the users will complete a post-use interview protocol that further 
elicits both specific and general reactions and comments. 

At the conclusion of usability testing, user comments and observer notes will be 
organized and reduced to a final set of identified problems and recommended solutions. 

Note that in addition to standard usability testing, the content of the materials will be 
analyzed for reading level. This analysis will use a standard computer program utilizing a 
broadly agreed-upon heuristic (such as the Gunning or Flesch) to assess reading level. 
The target is to developed TADS with a reading level not above high school graduate 
(excluding computer/technology specific words, which are necessary in training digital 
skills). 

B. Content validation . The goal of content validation is to determine whether the content of 
the TADS system reflects accurately and completely the knowledge and skill domains 
that it was designed to reflect. Thus, content validation is structured process that requires 
the judgment of subject matter experts as to appropriateness of content, given a particular 
domain. While the content validation process can be quantified, typically a qualitative 
approach is used, whereby reactions and judgments by experts are used to expose 
knowledge/skill content gaps, and domain-extraneous content. The degree of successful 
content development is then known, and shortcomings or errors can be addressed. 

TADS will be content validated for both the generic digital skills content, and for the 
Skills Bridge content. The generic digital skills content will be validated on the basis of 
input from digital skills trainers/teachers. These experts will enable us to verify that the 
general categories within the digital skills domain identified in Phase I (e.g., knowledge 
of computer system components, knowledge of operating systems) are complete and 
accurate. The second part of TADS content validation involves assuring that the 
AFATDS-specific information present in the Phase I prototype is accurate and complete. 
The experts required here will have expertise in the APATDS system. Note that this 
second level of content validation will continue throughout the full program 
development. 

Full Validation Study 

Although the training program development will include recurrent content validation throughout, 
it is still necessary that the entire training program be fully validated once it is complete. Below 
we explain why we feel that the training must be evaluated, then we discuss the proposed 
evaluation criteria and the evaluation design. 

Why digital skills training for AFATDS is a high priority for evaluation 

We are proposing an evaluation/validation of the digital skills training. Whenever an evaluation 
of training is considered, it is important first to understand whether an evaluation is feasible and 
appropriate. gOE (1998) surveyed people from over 100 organizations about “high priority 
evaluation” projects. Survey results showed that in projects where evaluation are considered 
critical: a) there is at least one clear reason or purpose for conducting the evaluation, b) the 
potential impact or scope of the service, program or initiative in question is high, and c) 
conditions that support the collection of evaluation data are favorable. 
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Each of these three conditions holds for the digital skills training for AFATDS. First, there are a 
number of clear reasons to conduct an evaluation of such training, including: a) its validation - to 
learn whether and why the training did or did not work, including gathering strong evidence 
regarding effectiveness (i.e., to collect evidence credible to the scientific community within the 
Army, b) to decide whether to continue/expand such digital skills training for all AFATDS 
trainees now in and the foreseeable future, c) to obtain diagnostic data in order to evolve digital 
skills training into its optimum form. Second, the potential impact of digital skills training is 
high, because a) the individuals participating in the training (soldiers) are critical to the nation’s 
defense readiness, b) there are large numbers of soldiers both within the AFATDS program and 
in other programs where digital skills are important, and c) the programs, such as AFATDS, that 
digital skills training might affect are expensive to develop, deliver, and maintain. Third, the 
conditions that support the collection of evaluation data are favorable because customers' 
expectations for the success of the service, program, or initiative are clear, the customer is 
interested in supporting evaluation, and the feasibility of gathering evaluation data in the military 
is high. 

Proposed Evaluation Criteria 

In order to best determine what criteria should be assessed, the general expectations of training 
should first be identified. We believe that, if digital skills training, combined with AFATDS 
training, is successful, then: 

• Trainees will increase their understanding of what is required of them to succeed 

• Trainees will be more ready to take on additional roles or rotate roles as required 

• Trainees' job performance will improve 

• Trainees' knowledge and skills will improve as a result of the training 

• Trainees will feel more confident in their ability to perform the job 

• Trainees' attitude toward digital jobs will improve 

• Trainees will feel training is meaningful 

• Trainees will be more able to learn other digital skills (the digital skills training will transfer 
to new situations) 

These expectations for what successful digital skills training will accomplish, along with the 
purposes of the evaluation (to collect credible scientific evidence relative to the training’s 
effectiveness, to help determine whether to continue/expand digital skills training, and to provide 
diagnostic information that will improve digital skills training) help define the appropriate 
training evaluation criteria. Table 2 provides an overview of planned evaluation criteria. Both the 
general nature of the criteria and the proposed collection method is specified. The rationale for 
some of these criteria, such as confidence (digital efficacy) was described in the discussion of 
Task 1. 
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Table 2: Proposed Evaluation Criteria 


Survey Supervisors, to assess: 

Whether unit retention/morale/attitude has improved. 

If trainees are ready to take on additional roles or rotate roles as required. 

If trainee performance has improved. 

Whether trainees have become more knowledgeable and skilled. 

Survey Trainees, to assess: 

If trainees have increased their understanding of what is required of them to succeed. 

If trainees feel ready to take on additional roles or rotate roles as required. 

If trainee job performance has improved. 

Whether they have become more knowledgeable and skilled. 

If trainee confidence about ability to perform job has improved. 

If trainee attitudes toward digital tasks have changed. 

Interview Supervisors, to assess: 

If trainees have increased understanding of what is required of them to succeed. 

If trainees are ready to take on additional roles or rotate roles as required. 

If trainee job performance has improved. 

Whether trainees have become more knowledgeable and skilled. 

Interview Trainees, to assess: 

If trainees have increased their understanding of what is required of them to succeed. 

If trainees are ready to take on additional roles or rotate roles as required. 

If trainee job performance has improved. 

Whether trainees have become more knowledgeable and skilled. 

If trainee confidence about ability to perform has improved. 

If trainee attitudes toward digital tasks have changed. 

Test Trainees, to assess: 

Whether trainees have developed digital skills. 


From the above table, it can be seen that Phase II will require the development of interview 
protocol, surveys, and tests. 


Evaluation Design 

The design for the proposed evaluation study is a pre, post, delayed-post treatment/ control 
group design. A design of this type does much to obviate a number of the so-called “threats 
to validity” (Cook & Campbell, 1979). The fact that a control group is included permits 
relatively strong conclusions about the unique effectiveness of differential treatment, which 
in this case is the digital skills training preceding the AFATDS training. Table 3 provides a 
summary of the planned design: 
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Table 3: Evaluation Study Design 
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Control Variables . There are a number of variables which it will be important to assess, in order 
that a) statistical control can be done if necessary (i.e., if the control group and the digital skills 
group differ systematically), and b) pre-existing differences among trainees may be tracked to 
see effects on training. These variables include g, general intelligence, and computer 
skills/experience. These variables will be assessed prior to training, using efficient group 
measures of each. 

Validation Sample. Part of this fourth task will be determining where we will get the sample of 
personnel to participate in the validation study. To minimize duplication of efforts, we will 
capitalize on the current AFATDS training as it presently occurs. At present, there are two ways 
that Field Artillery personnel gain the needed training on the AFATDS system: they either 
receive training as a part of their coursework while at Fort Sill or they receive training in the 
field with their entire battalion—we will pursue both venues as possible samples for the 
validation study. Beginning in September 2000, all field training for the AFATDS will be 
performed by Engineering and Professional Services, Inc. (EPS), an engineering firm that 
provides expert services in system communications technology. These field training exercises 
occur at multiple Army Bases multiple times throughout the year. They take place over a four 
week period, where the first week is set-up, and the second and third weeks are AFATDS 
training and the fourth week is “unit validation” where the soldiers do an exercise to make sure 
they can set up and use AFATDS in the field. EPS has agreed to support our Phase II effort by a 
variety of supporting activities such as helping us arrange access to the soldiers who will 
participate in the study, helping us choose a representative sample of the available personnel, 
helping us make the necessary arrangements to perform the validation study, and helping us 
remain in contact with the soldiers for follow-up data if needed. Another possible test bed is the 
US Army Reserve. With the split training option, recruits begin training during one summer 
(basic training) and finish it (specialization) the next. This would allow digital skills training to 
be given one summer, followed by IFSAS/AFATDS training the following summer. 

It is assumed that sufficient numbers of trainees will be available to allow for stable estimates of 
training effect size and other statistics such as correlation. A optimum sample size would exceed 
thirty trainees in each of the digital skills and control groups. 

Work Environment Readiness . In order to attain maximal understanding of the impact of 
digital skills training, it is necessary to understand the transfer environment. Conditions in the 
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field will support transfer of training to a certain degree and in certain ways. Similarly, various 
constraints on transfer will operate in any given training situation. Accordingly, in addition to the 
various effectiveness measures which will be developed and administered during Phase II, we 
will administer the Work Environment Readiness Survey, a validated diagnostic tool that can 
be distributed to trainees to determine their perceptions of the supportiveness of the learning 
environment (Tannenbaum, 1997). This survey includes items addressing such factors as 
“Learning Conditions” (factors that influence whether employees can successfully engage in 
continuous learning), and “Learning Sources” (trainees can leam and gain competence from a 
number of different sources). For example, the survey can identify whether a trainee has learned 
more from trial and error than any other single source. Similarly, the survey will be adapted to 
indicate the extent to which digital skills training influenced field success. 

A Measurement Model of Digital Skills Transfer for Phase II. Although we are examining 
one particular instantiation of the U.S. Army’s wide variety of tools demanding digital skills 
(AFATDS), and not measuring what might be termed the far transfer to an additional platform, 
the concept behind general digital skills training includes generalization to different tools. Figure 
10 illustrates this concept. 

The ovals in Figure 17 represent latent constructs which can be measured, with some degree of 
error, by visible or recordable indicators represented by rectangles. Arrows between latent 
variables signify influence while arrows between latent constructs and indicators represent 
measurement opportunities. The training interventions are shown as parallelograms. 
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Figure 17: A Measurement Model for Digital Skills Training 
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There are several points illustrated in the model. First, digital skills are considered a latent trait, 
observable through indicators or tests of performance. This suggests that it is desirable to 
measure digital skills in an abstract or general way, separate or separable from any specific tool. 
Second, digital skills training has an impact on both the trainable digital skills and on AFATDS 
training. Third, trainable digital skills influences field performance. Fourth, it is possible that 
there will be a relationship between level of performance on one task and that of another. Fifth, 
there are hypothesized immutable personal characteristics (such as g ) which affect performance 
on digital tasks. These are skills which will need to be “held constant” or “controlled” in the 
proposed studies. Not shown are trainable personal characteristics, such as self-efficacy or self¬ 
regulation, which can affect mastery and transfer of digital skills, but they can be seen as having 
the same influences (arrows) as trainable digital skills. 

7. Technical Objectives for TADS 

In Phase I we demonstrated the feasibility of developing a training program to increase 
adaptability in digital skills. In Phase II, we will develop the full training program and show 
evidence of its validity for training digital adaptability with the AFATDS system. 


Our goal in Phase II will be to develop a full training package that can be used to ensure that 
trainees get the background skills necessary to take full advantage of the digital revolution. We 
will design the training to prepare trainees to use a specific digital tool, the AFATDS. The 
general device knowledge module of the training will focus on that general knowledge needed to 
understand the AFATDS and other similar digital systems (e.g., computer systems in general, 
networking systems, data management systems); the device specific module will train the 
soldiers on how to map this general device knowledge to the specific AFATDS system. In 
addition to the training package, a related product of the second phase will be a description of the 
protocol necessary to revise this system-specific training module for other digital tools and 
situations. 

Phase II Objectives 

We have established four specific objectives to meet the Phase II challenge of developing a 
training program to increase adaptability in digital skills. The details of our approach for 
meeting each of these objectives are provided in Section 8, the Phase II Work Plan. 


Phase II- Objective h Completely map device knowledge to MOS specific tasks 


The core of our approach to training in adaptability, as developed in Phase I, is establishing links 
between generalizable knowledge about digital devices (e.g., the components of a network and 
the steps needed to set up a network) with information about a specific system (e.g., what 
networks are used in AFATDS and what steps are needed to set them up) and the specific goals 
to be accomplished (e.g., why an MOS 13D needs to set up an AFATDS network). Our review 
of the literature on transfer of training in Phase I suggested that establishing this type of linkage 
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between general knowledge, specific knowledge, and task goals is the best way to promote 
learning of adaptable skills. In Phase I, we selected the networking knowledge and skills 
required for AFATDS to demonstrate our training approach. The first step in Phase II will be to 
expand the approach demonstrated in Phase I to include additional aspects of AFATDS and 
additional MOS tasks. The product of this activity will be a mapping of basic digital skills and 
knowledge to device-specific AFATDS actions and to MOS 13D tasks. This mapping will be 
the foundation for Phase II training development. 


Phase II - Objective 2: Develop program to train adaptability in digital skills 


The majority of our activity in Phase II will focus on the design and development of the 
Adaptive Digital Skills Training package. We will follow the approach used in Phase I to 
develop storyboards for the training package, prepare a flowchart of the relationships between 
the training topics being covered, generate paper prototypes of the screens for review, develop 
electronic prototypes of the screens that can be reviewed on line, and prepare the final training 
package based on the prototype. The mode for delivering the training can be either CD or web- 
based, depending on the preferences of the Army. The advantage of CD delivery is that students 
can use the training without the need for an internet connection. The advantage of web-based 
delivery is that it is easily modified and updated at one central location. 


Phase //- Objective 3: Guarantee the validity of the training program 


The delivery of training that transfers from one problem or situation to another is a challenging 
task, and a critical objective for Phase II will be to conduct an evaluation study to assess the 
success of the adaptability training. The proposed innovative training approach has almost 
unlimited potential for use in other Army systems, other military services, and commercial 
environments, but, first, evidence is needed that it is effective. Considerable effort was focused 
in Phase I on the design of an evaluation study that will yield valid results. We have developed a 
set of proposed evaluation criteria, and will spend time during Phase II developing the measures 
necessary to determine the validity of the training program. The validation study we are 
proposing is a pre, post, delayed-post treatment/control group design. 


Phase 77- Objective 4\ Commercialize the digital skill training program 


The central goal of Phase II is to create a training program to increase adaptability in digital 
skills. While we plan to concentrate this effort on the MOS 13D and their use of the AFATDS, a 
portion of our efforts will be aimed at ensuring that the training to increase adaptability in digital 
skills has widespread applications for both military and commercial users. To enable this 
application diversity, we will solicit input from potential post application users early in the needs 
analysis, design, and validation process. In addition we will develop a demonstration 
methodology (via the Web) that can be used to cultivate a commercialization success. We will 
leverage both Aptima’s and gOE’s prior success in marketing the benefits of SBER research for 
application in real world systems (gOE’s history of developing and successfully marketing 
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commercial products and the details of Aptima’s most recent success in obtaining Phase III SBIR 
funding are discussed in the Section 9). 


The proposed commercialization will commence following the project kickoff meeting and 
continue throughout the entire Phase II period. Within the Army, the most immediate 
application is for the Intell school, where soldiers must stay current with frequently changing 
software. For non-DoD government applications, we see the digital skill adaptability training 
supporting the Department of Education’s lifelong learning programs and the Department of 
Labor’s programs designed to improve technology skills among adults. Outside the government, 
this training could be used in any job environment that requires the use of technology to ensure 
the ability of workers to keep up with quickly changing systems. Industry is currently facing a 
severe problem as workers must be retrained after the insertion of technological updates—the 
digital skill adaptability training could mitigate or alleviate this problem altogether. 


8. Plan for TADS 

We plan four interrelated tasks to develop the digital skill adaptability training and to collect 
validation data in Phase n. Figure 18 shows the planned schedule. The major tasks for Year 1 
will be the initial validation study of the training program developed in Phase I and the further 
development of the training program to begin in quarter 2 and to conclude by the end of the year. 
In Year 2 we will collect performance data to validate the training program in quarters 1 and 2. 
The final product, at the end of two years, will be 1) a program to train the general digital skills, 
knowledge, and procedures needed to understand the AFATDS system, and 2) written 
procedures for 
how to develop 
system-specific 
training for 
future training 
in different 
systems for 
different jobs. 

Plans for each 
of the four 
tasks, including 
a description of 
the technical 
approach for 
each task, are 
discussed in 
detail below. 


Tasks 


Year 1 Year 2 

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 



1. Conduct initial validation study 

2. Develop the training based on results of 

study; expand training v&fw 

3. Develop measures and conduct second 
validation study 

4. Further training development and final 
training package 


H— r 

Figure 18. Training Adaptability in Digital Skills Schedule for Phase II 


DAS WO 1-00-C-3014 

Training Adaptability in Digital Skills 


37 


AP-P-1128 



Aptima, Inc. 


Phase II Task 1: Conduct initial validation study 

This task has three parts: a) Conduct initial usability testing, b) Conduct initial content validation 
study, and c) Complete specifications for deliverable Digital Skills training system. The first two 
parts are described in detail in Section 6 of this report (“Design Validation Study”). The third is 
described below: 

Specification development . In addition to the initial validation studies, Task 1 requires 
that we complete specifications for the TADS system. 

In Phase I, we completed a simple prototype of the TADS system. In order for TADS to 
be scalable and possess sufficient flexibility, a number of factors additional to those 
considered for Phase I will be required of the final deliverable. Consequently, a most 
important step is the full specification of that deliverable. 

Some high-level requirements of the final TADS system include: extent and nature of 
database requirements, extent and nature of administrative capabilities, platform and 
computer system requirements, and design requirements such as color palette and 
visual/graphic themes. Note that, like the Phase I prototype, we envision a web-based 
system that enables both remote hosting and, if required, client or stand-alone platform 
installation. It is also anticipated that, unlike the Phase I prototype, the final system will 
be database capable, with administrative capabilities to enable non-HTML proficient 
users to tailor Skills Bridge information for systems other than AFATDS. That is, we 
expect to specify and create a product that has general applicability and is easily 
customized to provide maximum learning regardless of the digital system targeted. 

Below we identify in more detail the some of the work that will need to be done to 
complete the specifications for the Phase II deliverable. 

1. Administration. There should be at least two levels of administrative capability. 
First, local administrators should be able to log on and determine nature and extent of 
use of the TADS system. Information available to this administrator level could 
include user name, length of use, and user path/performance (e.g., number of 
assessment questions correct). A more complete user profile could be available that 
would include results of a pretest assessing initial level of digital skills. This kind of 
local administration capability would provide the ability to track who has completed 
TADS successfully, who has begun but not completed it, and who has not yet logged 
on to TADS. This facilitates local training management. In addition, individuals 
whose performance is problematic can be identified for remediation. 

A second level of administration would be available to those in charge of altering the 
content of the TADS system itself. This could include modifying the content of the 
basic digital skills training, and also adapting TADS with a new Skills Bridge for a 
new transfer environment. The figure below illustrated a mock-up of an 
administrative screen for adapting Skills Bridge content from AFATDS to a different 
environment (Radar). 
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TADS Administration 

On this page, input the Skills Bridge text that you want to appear 
for this Digital Skills topic. 


Skills Bridge Content 

Category: Computer 
Networks 

Topic: Cabling 

Learning Point: Tee 


2 Learning point applies 
to application: Radar 


Edit Content: 


Graphic URL: /networks/cables/tee.gif 


Preview Graphic 



Tee connectors are used in the 
AFATDS system because AFATDS 
stations are often configured as a bus 
network. 



Previous Learning Point Next Learning Point 


Figure 19: An administrative interface for Skills Bridge adaptation 

These kinds of administrative capabilities are crucial in producing a deliverable that 
has the greatest value to the widest number of possible transfer environments. As 
explained elsewhere, the Skills Bridge integrates the general digital skills content into 
a specific application. 


2. Database and Information Processing Systems. Because of the desirability for 

make TADS to be a dyanamic, database-driven system with administrative 
capabilities, it is necessary to specify the nature of that system. In its simplest term, 
TADS will be a web-based application. In such a system, the browser sends a request 
to the internet information server, which processes the request, obtains any 
information required from the database, and then serves out the final information (in 
the form of a web-page) to the browser. This is illustrated in the picture below. 



Figure 20: How data is transferred to browser 
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For example, the administrative capabilities already mentioned are made possible 
through the access of database-stored information, and operate via special 
administrative pages (like the one illustrated in Figure 19) that send the appropriate 
information and requests to the server. 

Thus, another critical set of specifications relate to the nature and structure of the 
system databases. Also required is a final determination of the systems (e.g., Oracle 
or SQL Server) and languages (e.g., Java or ASP/VBScript) used for developing the 
active pages of the TADS system. 

Task 2. Develop and expand training based on the results of the validation study 

Based on the results of the initial validation study, we will develop and expand our training 
program. This will entail mapping general digital knowledge and skills to the AFATDS tasks; 
Designing the training for general device knowledge; and Developing the training for general 
device knowledge. 

Map general digital knowledge and skills to AFATDS tasks 

We will expand the approach demonstrated in Phase I (i.e., mapping general digital knowledge 
to setting up computer networks in the AFATDS system) to include additional aspects of 
AFATDS and additional MOS tasks. The product of this activity will be a mapping of basic 
digital skills and knowledge to device-specific AFATDS actions and to MOS 13D tasks. This 
mapping, which will be the foundation for Phase II training development, will take place in the 
following two subtasks. 

Subtask 2.1: Continued identification of general knowledge, skills, and human characteristics 
that underlie both successful adaptation to and learning of digital tasks in general. 

Phase I work in identification of knowledge, skills, and human characteristics relating to digital 
performance will be augmented and extended. In particular, the goal will be to identify and 
specify two domains: a) general digital knowledge and skills, and b) specific human 
characteristics related to success in learning and performing digital tasks. 

This first domain, general digital knowledge and skills, was partially identified in Phase I work. 
In essence, the idea of general digital knowledge and skills is simple. There are basic general 
concepts and skills a) that can be taught, and, b) that once learned can improve further learning 
on specific tasks that load heavily on digital skills. For example, advance organizers, device 
knowledge, and schema/prototypes are all related, overlapping descriptions for learning 
preparation as researched by cognitive and learning psychologists. The literature is replete with 
references to models of human learning that include components of learning “readiness”. This 
literature will continue to be mined in the early stages of Phase II. 

Other general skills that are important to AFATDS in particular will be identified. For example, 
AFATDS performance requires some understanding of maps and mapping and artillery 
geometry. These are general skills that, while not critical to the training of many digital skills, 
are important in this instance. It seems important to identify such other general skills so that they 
can receive the same instructional design treatment as general digital skills and be included in 
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training. The identification of such other general skills will then be part of the final delivered 
process for adapting digital skills training for any particular situation. 

Also in the early part of Phase II, the second identified domain will be further specified to assist 
in completing Phase II goals. This domain is that of specific human characteristics related to 
success in learning and performing digital tasks that can be influenced by training. Most 
important among these characteristics is what we term “digital efficacy” - a generalized sense of 
competence that predicts learning digital skills. Research has shown repeatedly that confidence 
in a task relates to performance. Self-efficacy (Bandura 1977, 1984) is a domain-specific, 
focused belief in the ability to perform a specific task or in a specific arena of activity. Self- 
efficacy can be critical for good performance - Bandura has even suggested that self-efficacy 
will “usurp the lion's share of the variance in human conduct” (Bandura, 1984, p. 252). The 
practical implication is that in digital skills training, it will be important to foster digital skills 
self-efficacy. Other research has confirmed that self-efficacy has a strong influence on many 
aspects of individual performance, both in and out of the classroom. 

In a field study of USAF aircraft technician (“Mission Ready Technician” or MRT) training, 
Alliger, Bennett, Eddy, & Tannenbaum (in press) found that task self-efficacy (as reported by the 
technician) strongly predicted task performance (as reported by the supervisor). These were field 
measures. Although self-efficacy was somewhat more restricted in range for F-16 MRTs than for 
C-141 MRTs, the same general relationship between performance and confidence held. This 
relationship in both cases was a remarkably strong one (r’s - .68, .86). Gist (1989) demonstrates 
that the method of training can be critical to the degree of trainee self-efficacy. There also 
research that training method is related to self-efficacy in the specific area of digital skills (e.g., 
Gist, Schwoerer, & Rosen, 1989). Accordingly, the successful inculcation of “digital efficacy” is 
one critical goal of the proposed digital skills training. 

In Phase I we focused our training development on Computer Networks. Other general digital 
knowledge and skills that are candidates for the full training program include: 

• File Management: Copying Directories, Copying Files, Creating Directories, Deleting 
Directories, Deleting Files, Directory Tree, Moving Directories, Moving Files, Renaming 
Directories, Renaming Files, Root 

• GUI: Cursor, Desktop, Double-Click, Drag And Drop, Hot Keys/Shortcut Keys, Icons, 
Multi-Tasking, Point And Click, Right-Click 

• Hardware: Cd-Rom, Keyboard, Modem, Monitor, Motherboard -CPU, Mouse, Powering 
Up, Printer, Scanner 

• Memory: RAM, ROM 

• Software: Database, E-Mail, Graphics, Other, Spreadsheet, Web Browsers, Word 
Processing 

• Storage: Cd-Rom, Floppy, Hard Drive, Sizes (Bits, Bytes, KB And MB), Zip 

• Operating System: Control/Utility Panels, Find/Search, Help, Menus, Reboot, Recycle 
Bin, Shutdown, Starting, Managing Tasks, Troubleshooting Tools 
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Part of this Phase II subtask will be accomplished through completion of the literature review 
work started in Phase I. It is also foreseen that interviews with both expert and below-average- 
performing AFATDS operators will help us understand what digital knowledge, skills, and other 
characteristics differentiate the experts from the inferior-performing soldiers. In other words, if 
we can identify what base knowledge is possessed by an individual who succeeds in training 
(becomes an expert), and if there are clear differences in this regard between experts and those 
that fail to become good AFATDS performers, then those differences should provide guidance in 
terms of the skills and knowledge that are important pre-requisites to successful training. This 
could be called expert profiling. 

Subtask 1.2: Mapping of Digital Skills and Device Knowledge to AFATDS tasks . 

This task requires the general digital skills and device knowledge identified in subtask 1.1 to be 
mapped to tasks learned in AFATDS training. In a sense, this is a content-validation strategy. 
Subtask 1.1 completed the identification of general digital skills. Not all of these will apply to 
any specific training program, including AFATDS. Nor, even in AFATDS, will general digital 
skills underlie all aspects of performance. Consequently, it will be necessary to understand in 
detail how general digital skills map onto AFATDS tasks. Figure 21 illustrates the expected 
outcome. 



General digital skills that map onto AFATDS tasks 


Figure 21: How general skills underlie AFATDS tasks 

Therefore, given that many trainees do not have a background in computers or the kind of digital 
skills that would prepare them to face the potentially intimidating topics included in AFATDS 
training, we believe, as outlined in the introduction of this proposal, that a preliminary training in 
digital skills will benefit trainees. However, given the vast area of digital knowledge that could 
be trained, the question is: how to correctly specify which topics will be most appropriate for 
AFATDS training? 

Like many modem military systems AFATDS is highly computerized. Consider some of the 
terms and concepts that trainees face in mastering AFATDS (AFATDS Operator’s Notebook, 
1999): 
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10 Base 2 

• Debugging 

• Parallel, Parallel Port 

15-pin Receptacle 

• Distributed 

• Router 

40-pin Port 

• Domain Name Server 

• SCSI 

AC/DC Power 

• Drive, Drive Access 

Door 

• String 

• Subnet, Subnet 

Accelerator Keys 

• Hostname 

masking 

Access Privileges 

• Initializing 

• System Administration 

Bandwidth 

• Internal LAN, External 

• TCP/IP 

Boot/Reboot 

LAN, WAN 

• Unix, Xterm 

Byte 

• IP Address 

• UPS Power Strip 

CD, CD-ROM, CD- 

• LAN Dust Cover, LAN 

• Workstation, Multi- 

Caddy 

Terminator, LAN Tee 

Workstation 

Coax, Coaxial 

• Login, Password, 


Database, Database 

Password Confirmation 


Backup, Database 

Utilities 

• Menus, Menu Options 

• Optical Disk 



A number of general areas of knowledge can be perceived when the above list is studied. 
“Knowledge of computer networks, including hardware and configuration” for example, is one; 
another is “Knowledge of databases.” A third is “Knowledge of computer components.” We 
propose a two-step process for the task of understanding how to map these general areas onto 
AFATDS training: a) SME content-validation, and b) mapping of knowledge and skills to 
AFATDS domain. 

The content validation and mapping will occur as follows. In Phase I, we identified several 
potential AFATDS instructors who expressed a willingness to participate as subject matter 
experts to assist in the mapping task described in this section. These individual will be requested, 
via proper US Army channels, to participate in a content validation exercise, possibly in a 
facilitated workshop that will be held at Ft. Sill. This SME exercise will accomplish much of the 
content validation identified in Task 1 B. In addition and as a parallel process, the subject matter 
experts will assist us in mapping general digital skills onto AFATDS tasks. 

Content validation by AFATDS SMEs . Using the domain of digital skills which we have 
defined in Phase I, we will present the AFATDS SMEs with of a master list of digital skills, and 
obtain agreement on which of these skills load (and to what degree they load) onto these 
AFATDS instructional components. In this way, it will be possible to correctly identify and 
prioritize the general digital knowledge and skills that underlie successful learning of AFATDS 
skills. This assures content validation of the digital skills training via standard content validation 
techniques. 

Content-Validation Survev/Interview of AFATDS SMEs. A survey-type protocol will be 
developed that will list all components of AFATDS instruction (e.g., Preparing AFATDS for 
Operation, Implementing AFATDS Unit Database), and that will request respondents to indicate 
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which of a master list of digital skills load (and to what degree they load) onto these AFATDS 
instructional components. In this way, it will be possible to correctly identify and prioritize the 
general digital knowledge and skills that underlie successful learning of AFATDS skills. The 
AFATDS Subject Matter Experts (SMEs) that will be asked to participate in this survey may 
include both civilian and U.S. Army AFATDS experts, such as instructors and supervisors. 
Analysis of the survey results will proceed in a straightforward manner: computation of average 
SME general skill ratings for each instructional component. 

Map General Digital Knowledge and Skills to AFATDS Tasks . The data from subtask 1.1 
will result in a prioritized list of general digital skills, mapped onto the AFATDS curriculum. For 
example, something like the following might result: 


Table 4: Example of Mapping 


AFATDS Training Component 

Important Aspects of Digital 
Skills Training (in order of 
importance) 

Other Important General 
Knowledge and Skills 

Identify and Cable AFATDS 
Components 

General Network Knowledge, 
Cabling, Cabling Connectors, 
Distributed Systems 


Prepare AFATDS for Operation 

General Network Knowledge, 
Cabling, Cabling Connectors 


Implement AFATDS Mapping 

Displays, Menus 

Mapping Knowledge (e.g., 
Coordinates, Scales) 

Implement AFATDS Battlefield 
Geometry 

Displays, Menus 

Artillery Geometry 

Hardware and Software 
Communications Configuration 

Communications Hardware and 
Software, Distributed Systems 

Communications Protocol 

Implement AFATDS Data 
Distribution 

Databases, Distributed Systems, 
General Network Knowledge 


Database Trouble-shooting 

Databases, Digital System 
Troubleshooting 

Troubleshooting 


To consider a single example in more detail, the concept of networks is addressed. AFATDS 
operates as a 10 Base 2 configured Local Area Network (LAN). It operates over coaxial cables 
with BNC connectors, achieving transmission rates of 10 millions bytes per second. In AFATDS 
training, users are taught how to set up the hardware for such a network, including cabling and 
connectors. However, a LAN is a general concept; the current AFATDS is simply one 
instantiation a LAN, just as a LAN is a particular type of network. Therefore, we can say that the 
general concept of a computer network “maps” onto AFATDS, as Figure 22 indicates: 
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Networks 




WANs (wide area networks) 


^ MANs(metropolitan area networks) 

LANs (local area networks) 

► Components (e.g., router, computers/workstations, hub, server) 
Characteristics (e.g., reliability, throughput, security, access) 
w Protocol (e.g., TCP/IP, SMTP) 


Operating System (e.g., Unix, Linux, Windows) 


AFATDS as particular example of a LAN 


Components: Ethernet hub/routers, workstations 
Characteristics: Reliable, very secure, fast 
Protocol: TCP/IP 
Operating System: Unix 


Figure 22: How the general concept of networks is instantiated in AFATDS 


It is planned to systematize this content-validation survey approach for the final, portable digital 
skills training process which is envisioned as a central outcome of Phase II. 

Design the training for general device knowledge 

In an extension of our Phase I effort to understand the training requirements for digital mastery, 
we will gather more detailed information about how EPS trains soldiers to use AFATDS and 
what complex skills are being addressed. Specifically, a team of two psychologists will 
participate in one of the AFATDS training regimens alongside soldiers receiving the instruction 
for the first time. For two weeks, our team will gather data on the content of the AFATDS 
instruction, the methods used, and the responses of students to the training. We will focus on 
questions such as “What units are the most difficult for students?” and “What are some useful 
exercises we can draw from the material”? and “Are there particular student groups emerging 
during the training as a result of the demands placed on them? For example, can you tell by the 
teacher-student interactions or student conversations which students are already skilled and 
which have no internet experience?” This data will give us invaluable and otherwise inaccessible 
information about the cognitive skills we need to train and the most relevant, instructive 
examples to import into our training. The data will heavily influence the development of all three 
training modules—the device model, procedural model, and Skills Bridge. 

It is also during this second task of our SBER. Phase II that we will determine what the overall 
appearance of the training will be and how the entire training package will flow. These two 
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aspects of the training may seem trivial, but they are actually integral parts of the training that 
have a great affect on the overall effectiveness of the program itself. If the flow of the training 
program is counterintuitive, trainees will get lost in the maze of the training screens, become 
frustrated, and will gain nothing from the training program. Likewise, if the appearance of the 
training screens is inappropriate for the intended audience, or if the screens are too cluttered or 
too sparse (e.g., lack important information), the trainees will not get the full benefits of the 
training. To ensure the proper development of the training program, we will begin with 
storyboarding, graduate to detailed flowcharts and paper prototyping, then do actual electronic 
prototyping before developing the final product. 

Storvboarding . This initial phase of web-based training development is similar to 
brainstorming to generate ideas about how to develop the training. In this stage of the training 
development, suggestions are made about the design of the training package, these suggestions 
are discussed, and ideas for the training program are sketched out to prepare for further 
development. 


Flowchart . The flowchart is similar to a road map for the training program that illustrates how 
the multiple screens will be linked together in the final product. By creating a flowchart before 
developing the final product, the designer is forced to think of all the many implications of the 
product design and therefore make best use of the links to maximize training effectiveness. An 
example of a simplified flowchart developed for the Phase I Network Training program can be 



Figure 23: Flow of the TADS Training Program 
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Paper Prototyping . As the flowchart is being prepared, paper prototypes of each screen will 
also be generated so that people can use both the paper prototypes and the flowchart to become 
familiar with the plan for the training program and make suggestions for improvement. 

Electronic Prototyping . Once a satisfactory design has resulted from the paper prototyping 
stage, an electronic prototype will be assembled to allow people to sit at a computer screen and 
progress through a sample of links. This stage of the training design will overlap with the actual 
coding of the training program so advice resulting from people navigating through the electronic 
prototype can be fed directly into the final product code. 

These processes will enable us to get input and advice from training experts, developers, and 
potential trainees before spending the extensive hours needed to fully develop the training 
program. 

Develop the training for general device knowledge 

There are several optional mechanisms for delivering the instruction and the evaluation, any of 
which we can implement based on the Army’s preferences. One method is server-side delivery 
in the form of a web-based instructional program. The advantage of web-based instruction is the 
control that trainers have over the content and its accessibility. When instruction is server-side, 
the developers control which version of instruction is available and can update or de-bug 
instruction for all students at once without the sometimes difficult-to-get involvement of the 
students. Another method is for the instruction and evaluation to be client-side in the form of a 
CD-ROM. The advantages of a CD are that students who do not have access to the internet can 
still access the instruction on their own PC anytime. Regardless of the manner in which the 
training is delivered, it will likely be written in HTML (looking something like the screen shots 
in Section 5 above) so the students can learn at their own pace and with minimal trainer input. 

In addition to the self-paced web-based or CD-ROM training, text-based lesson plans for 
classroom presentations may be useful for the Skills Bridge module and for some testing 
situations, such as late post-testing after a student has completed comprehensive instruction in a 
particular system. Lesson plans that instructors can print out and use in their classrooms are 
useful whenever the instructor needs to have control of the class. This is why the Skills Bridge 
module may be presented to students as a classroom-based lecture and exercise unit. In this case, 
a large part of the content is information about a specific system—information that the instructor 
or training developer controls. In cases where an evaluation is needed to assess the impact of the 
device and procedural training, a post-test could be developed by the instructor with guidance 
from the proposed training package. 
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Task 3. Conduct second validation study 

Once we have fully developed the training program, we will conduct a validation study to 
determine how effective the program is in increasing a trainees adaptability in digital skills. This 
validation study is fully detailed in Section 6 of this report (“Design Validation Study”). 

Task 4. Further training development and final training package 

Based on the results of the validation study, we will refine the general digital skills training 
program and deliver a final package that can be used to train general digital skills for any number 
of computer systems. It is in this final task that we will make sure the program works smoothly 
and that no special training or knowledge is needed to operate it. 

In addition to the digital skills training program, we propose to provide a manual explaining the 
procedure used to develop the system specific training. The true benefit of the training program 
proposed in this SBER is the way that it uses the general device knowledge and general procedure 
knowledge to bridge the gap between the job’s tasks and the specific actions necessary to 
perform those tasks. Most of this training involves the general knowledge found in the general 
digital skills training program, however it also requires specific knowledge to link the general 
skills with the specific job tasks and behaviors—this part of the training must be developed each 
time the training is to be implemented in a new job or system. The process for developing this 
part of the training will remain the same regardless of the specific job tasks or behaviors; so part 
of our final product will be a well documented process, in the form of an electronic manual, that 
will walk training developers through the procedures necessary to develop this part of the 
training program for any job or system. 

9. Commercialization of TADS 

A growing market: According to Business Week (August 3, 2000), the corporate e-leaming market in 
the U.S. topped $550 million in revenue in the year 2000, and is expected to grow to $11.4 billion by 
2003. The technology for a program to develop adaptive digital skills that the Aptima/gOE team 
is developing under this SBIR project has the potential to tap this quickly expanding market. 
Although, initially, the content and training strategies will be tailored for ARI’s military 
customers, the sound underlying pedagogical principles, basic technology, and unique construct 
of the training package make it an ideal candidate for commercialization in several 
technology/market pairs in the general training market. 

A commitment to transition and commercialization: Both Daniel Serfaty, Aptima’s principal 
founder and Chief Scientist, and Scott Tannenbaum, gOE’s founder, have a proven record of 
spinning off and marketing the technological ideas, methods, and products emerging from the 
R&D activities in both their companies. Daniel Serfaty is personally committed to apply his 
entrepreneurial skills to lead the project team in pursuing an aggressive commercialization 
strategy for the Training Adaptability in Digital Skills product. For example, working from a 
current SBIR Phase II project for AFRL, Aptima has been able to add almost $1.5 M of 
additional funding ($500K in supplementary funds from the Air Force and other customers 
interested in supporting the technology, as well as an additional $1,000K in Phase in funding 
directly from the AWACS program manager’s office.) This vision, combining technical 
excellence with a focused marketing approach, is what has allowed Aptima to grow 
exponentially over the last four years into a $4M/year human-centered engineering company for 
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both commercial and government customers. This market orientation has also enabled gOE to 
develop into a market leader in training programs for a very wide range of clients. Both founders 
are taking it upon themselves to collaborate in developing and implementing a strategy for Phase 
III transition and commercialization. 

There are numerous commercialization opportunities in both government and industry for a 
training program that trains general principles of digital skills. We intend to target the Training 
Adaptability in Digital Skills tool to both government and commercial customers in both a 
horizontal and a vertical market expansion. 

Horizontal Commercialization: Similar Technology-Wider Markets: First we are focusing on 
the horizontal marketing of the Training Adaptability in Digital Skills technology. By horizontal 
market expansion, we mean expansion into different markets that require few changes in the 
overall technology. Some examples of horizontal expansion are: 

• Training Adaptability in Digital Skills for the Army intelligence community. Soldiers in this 
area must constantly learn to use new versions of digital tools 

• Training Adaptability in Digital Skills for TRADOC in the Army XXI and the Objective 
Force. One of the changes in the Army for the Objective Force is an increase in digital tool 
use, as well as quicker transitions from one digital tool to the next as upgrades occur more 
quickly. This necessitates the need for training in adaptability. The Training Adaptability in 
Digital Skills tool could very easily be used to train soldiers to guarantee that the Army of 
today will be ready for tomorrow. 

• Training Adaptability in Digital Skills for civilian safety personnel (e.g., fire and police 
officers, disaster relief personnel) to ensure they are prepared for the future. The Military is 
not the only profession faced with rapid change in the near future as a result of the influx of 
technology. Civilian jobs such as the job of police officer are experiencing similar changes, 
and they must be prepared to deal with the changes as they occur. The Training Adaptability 
in Digital Skills tool could easily be modified to ensure that our civilian safety personnel are 
prepared for the future. 

• Training Adaptability in Digital Skills in other civilian jobs that will be increasingly 
impacted by technology such as manufacturing and clerical work. Technology is not 
confined to military and safety personnel, all vocations will be affected by it. The 
Department of Labor forecasts the skills that will be required over the next 20 years in order 
to plan its programs for adult education and retraining, and the insertion of new technology 
into the workplace is a large driver of changing skill demands. The Department of Education 
is also interested in forecasting the demand for adult education driven by changing jobs. 
Because the Training Adaptability in Digital Skills tool is a training program that focuses on 
general principles of technological skills, it could be used to train the employees in 
manufacturing and clerical jobs to prepare them for the quickly changing future. 

Vertical Commercialization: Diversified Technology-Similar Market: We are also exploring 
possible vertical expansion of the Training Adaptability in Digital Skills technology (i.e., we are 
looking into ways that we can expand Training Adaptability in Digital Skills by adding different 
technologies). Some possible vertical expansion opportunities are: 

• Expanding from training to selection tests. The Training Adaptability in Digital Skills tool 
could very easily be modified in such a way that it could be used to select employees who are 
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adaptable. Companies are realizing that adaptability is a skill crucial to the organization’s 
success. If organizations could select individuals who are able to adapt quickly, they could 
ensure success in a constantly changing environment. 

• Expanding from training to training assessment. The Training Adaptability in Digital Skills 
tool could also be modified to assess training needs. Individuals could be tested to determine 
their “adaptability quotient,” and training could be recommended for those individuals who 
score below some acceptable threshold. 

In Summary, we see a rapidly expanding market for the Training Adaptability in Digital Skills 
tool. It fills a gap created by the swiftly increasing technology-based market. It is no longer 
possible to keep up with technological changes in a reactive way - implementing training after 
the technology has been introduced. Now the only way to succeed is to prepare pro-actively for 
technological change before it occurs. Training Adaptability in Digital Skills is a tool that will 
allow military and civilian organizations to prepare for the future. 
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Appendix A: 

Results of AFATDS Training Student Survey Administered Week of 
10.16.00 at Net Team Training, Fort Sill 
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AFATDS Training Student Survey 

This survey will allow researchers from Aptima, Inc to gather information needed to develop 
supplemental AFATDS training - it will NOT be used to evaluate the AFATDS students, trainers, 
or the training program. Please do not put your name on the survey; all answers will be held in 
the strictest of confidence and the information will be used only in the development of 
supplemental training. ___ 

Please answer the following questions: 

Rank/MOS/DMOS ___ 

Flow many months/years of experience do you have with each of the following? (If none, leave 
blank) 


years_ 

months with IFSAS 

years 

months with FDS 

years_ 

months with TACFIRE 

years 

months with DMD 

years_ 

months with BCS 

years 

months with a Personal 

years 

months with AFATDS 

Computer 



Flow much experience have you had with a windows-type computer operating environment 
(such as Microsoft windows)? (circle appropriate number) 


1- 

None 

-2- 

-3- 

-4- 

some 

-5- 

-6- 

-7 

a lot 

How much experience have you had with computer networks? 



1- 

-2- 

-3- 

-4- 

-5- 

-6- 

-7 

None 



some 



a lot 

How much experience have you had with data input and manipulation? 


1- 

-2- 

-3- 

-4- 

-5- 

-6- 

-7 

None 



some 



a lot 

How much experience have you had with computer games? 



1- 

-2- 

-3- 

-4- 

-5- 

-6- 

-7 

None 



some 



a lot 

Before the AFATDS training, how comfortable were you with computer systems? 


1- 

-2- 

-3- 

-4- 

-5- 

-6- 

-7 


Not at all moderate very comfortable 

(I had very little (I had an email address (I had experience 

exposure to computers) and surfed the web) programming computers) 

In your opinion what most helped you learn from the training? (examples: understanding the 
terminology, understanding Field Artillery) 


What do you think might help you learn more from the training? (examples: better 
understanding of the terminology, better understanding of Field Artillery) 


Other comments or questions that could help us develop supplemental training: 
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Results of AFATDS Training Student Survey (Raw Data) 

Administered Week of 10.16.00 at Net Team Training, Fort Sill 


RANK 



Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Valid 

3 

6.4 

6.4 

6.4 

13P10 

1 

2.1 

2.1 

8.5 

E-3 

1 

2.1 

2.1 

10.6 

E-4 

2 

4.3 

4.3 

14.9 

E-5 

1 

2.1 

2.1 

17.0 

E4 

5 

10.6 

10.6 

27.7 

PFC 

6 

12.8 

12.8 

40.4 

PVT 

1 

2.1 

2.1 

42.6 

SFC 

1 

2.1 

2.1 

44.7 

SGT 

8 

17.0 

17.0 

61.7 

SPC 

9 

19.1 

19.1 

80.9 

SSG 

8 

17.0 

17.0 

97.9 

SST 

1 

2.1 

2.1 

100.0 

Total 

47 

100.0 

100.0 



MOS 



Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Valid 

4 

8.5 

8.5 

8.5 

13 

1 

2.1 

2.1 

10.6 

1310 

1 

2.1 

2.1 

12.8 

13P 

30 

63.8 

63.8 

76.6 

13P10 

1 

2.1 

2.1 

78.7 

13P20 

2 

4.3 

4.3 

83.0 

13P2P 

1 

2.1 

2.1 

85.1 

13P30 

2 

4.3 

4.3 

89.4 

13P4H 

1 

2.1 

2.1 

91.5 

13PAPA 

1 

2.1 

2.1 

93.6 

E5 

1 

2.1 

2.1 

95.7 

SPC 

2 

4.3 

4.3 

100.0 

Total 

47 

100.0 

100.0 
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DMOS 



Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Valid 

35 

74.5 

74.5 

74.5 

13P 

3 

6.4 

6.4 

80.9 

13P10 

2 

4.3 

4.3 

85.1 

13P20 

i 

2.1 

2.1 

87.2 

13P30 

2 

4.3 

4.3 

91.5 

13P40 

1 

2.1 

2.1 

93.6 

13PAPA 

1 

2.1 

2.1 

95.7 

30 

1 

2.1 

2.1 

97.9 

35309 

1 

2.1 

2.1 

100.0 

Total 

47 

100.0 

100.0 



Descriptive Statistics 







Std. 


N 

Minimum 

Maximum 

Mean 

Deviation 

Experience on IFSAS 
(yr.mo format) 

10 

.10 

4.00 

1.0900 

1.3345 

Experience on TACFIRE 
(yr.mo format) 

5 

1.00 

5.00 

3.4000 

1.6733 

Experience on BCS 
(yr.mo format) 

8 

.10 

13.00 

5.1375 

4.6577 

Experience on AFATDS 
(yr.mo format) 

11 

.10 

.60 

.1909 

.1814 

Experience on FDS 
(yr.mo format) 

40 

.10 

12.00 

3.6962 

3.1256 

Experience on DMD 
(yr.mo format) 

3 

.90 

2.00 

1.3000 

.6083 

Experience on Personal 
Computer (yr.mo format) 

36 

.10 

15.00 

5.1667 

4.0029 

Valid N (listwise) 

0 
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How much experience have you had with a windows-type computer 
operating environment (such as Microsoft windows)? 



Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Valid none 

3 

6.4 

6.5 

6.5 

2.00 

4 

8.5 

8.7 

15.2 

3.00 

2 

4.3 

4.3 

19.6 

some 

12 

25.5 

26.1 

45.7 

5.00 

9 

19.1 

19.6 

65.2 

6.00 

3 

6.4 

6.5 

71.7 

a lot 

13 

27.7 

28.3 

100.0 

Total 

46 

97.9 

100.0 


Missing System 

1 

2.1 



Total 

47 

100.0 




How much experience have you had with computer networks? 



Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Valid none 

8 

17.0 

17.4 

17.4 

2.00 

6 

12.8 

13.0 

30.4 

3.00 

4 

8.5 

8.7 

39.1 

some 

12 

25.5 

26.1 

65.2 

5.00 

8 

17.0 

17.4 

82.6 

6.00 

3 

6.4 

6.5 

89.1 

a lot 

5 

10.6 

10.9 

100.0 

Total 

46 

97.9 

100.0 


Missing System 

1 

2.1 



Total 

47 

100.0 




How much experience have you had with data input and manipulation? 



Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Valid none 

6 

12.8 

13.0 

13.0 

2.00 

8 

17.0 

17.4 

30.4 

3.00 

10 

21.3 

21.7 

52.2 

some 

8 

17.0 

17.4 

69.6 

5.00 

3 

6.4 

6.5 

76.1 

6.00 

4 

8.5 

8.7 

84.8 

a lot 

7 

14.9 

15.2 

100.0 

Total 

46 

97.9 

100.0 


Missing System 

1 

2.1 



Total 

47 

100.0 
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How much experience have you had with computer games? 



Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Valid none 

4 

8.5 

8.7 

8.7 

2.00 

5 

10.6 

10.9 

19.6 

3.00 

1 

2.1 

2.2 

21.7 

some 

10 

21.3 

21.7 

43.5 

5.00 

11 

23.4 

23.9 

67.4 

6.00 

5 

10.6 

10.9 

78.3 

a lot 

10 

21.3 

21.7 

100.0 

Total 

46 

97.9 

100.0 


Missing System 

1 

2.1 



Total 

47 

100.0 




Before the AFATDS training, how comfortable were you with computer systems? 



Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Valid Not at all (1 had very little 

3 

6.4 

6.5 

6.5 

exposure to computers 





2.00 

2 

4.3 

4.3 

10.9 

3.00 

7 

14.9 

15.2 

26.1 

moderate (1 had an email 





address and surfed the 

17 

36.2 

37.0 

63.0 

web) 





5.00 

5 

10.6 

10.9 

73.9 

5.50 

1 

2.1 

2.2 

76.1 

6.00 

6 

12.8 

13.0 

89.1 

very comfortable (1 had 





experience programming 

5 

10.6 

10.9 

100.0 

computers) 





Total 

46 

97.9 

100.0 


Missing System 

1 

2.1 



Total 

47 

100.0 




Descriptive Statistics: Rating Questions 



N 

Minimum 

Maximum 

Mean 

Std. 

Deviation 

How much experience have you had with a windows-type computer 

46 

1.00 

7.00 

4.7609 

1.8641 

operating environment (such as Microsoft windows)? 






How much experience have you had with computer networks? 

46 

1.00 

7.00 

3.7609 

1.8995 

How much experience have you had with data input and manipulation? 

46 

1.00 

7.00 

3.7391 

1.9713 

How much experience have you had with computer games? 

46 

1.00 

7.00 

4.6087 

1.8912 

Before the AFATDS training, how comfortable were you with computer 

46 

1.00 

7.00 

4.2935 

1.5937 

systems? 






Valid N (listwise) 

46 
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Case Summaries, Comment 1 



In your opinion what most helped you learn from the training? (examples: 
understanding the terminology, understanding Field Artillery) 

1 

Having previous experience w/MS Windows & the operating style. 

2 

1 have previous training w/Windows. 

3 

More detailed computer system classes/understanding 

4 

Field problems, warfighter exercise. 

5 

Practical exercise. 

6 

The hands on PE's. 

7 

Trying to understand AFATDS how it runs. 

8 

PE 

9 

Understanding Field Artillery & training 1 have had. 

10 

Terminology 

11 

Understanding terms, FA working w/FCS & MS Win. 

12 

Hands on training. 

13 

The PE's were very helpful in my learning of AFATDS. 

14 

Hands on. 

15 

Understanding Field Artillery. 

16 

Understanding FA. 

17 

The system itself. 

18 


19 

Understanding FA. 

20 

Photographic memory. 

21 

PE 

22 

The PE's. 

23 

How confusing AFATDS is/don't think 1 learned anything new, just more windows. 

24 

Neither, understanding how to use a PC. 

25 

Being familiar with FA and having computer experience. 

26 

Hands on. 

27 

In depth of mission planning and strategies. 

28 

Understanding Field Artillery. 

29 

Hands on during class. 

30 

I understand that there is a lot more to it than what 1 thought. 

31 

I've been it the Field Artillery for over 2 years (experience of Fire Direction). 

32 


33 

Already having experience in fire direction. 

34 

Hands-on exercises. 

35 

The hands-on experience. 

36 

The exercises the class had to complete. 

37 

Understanding basic computers and understanding Field Artillery. 

38 

Training exercises, hands on training. 

39 

The terminology and the window setups. 

40 

Practical exercises. 

41 

Understanding Field Artillery, computer skills. 

42 

Understanding how this s/w applies to F.A. 

43 

Doing the P.E. 

44 


45 

Hands on training. 

46 

All in doing. 1 now have a better understanding of my job as 13P. 

47 

Hands on working w/the system. 
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Case Summaries, Comment 2 



What do you think might help you learn more from the training? (examples: better 
understanding of the terminology, better understanding of Field Artillery) 

1 

Slower training 

2 

Need a looser environment. 

3 

More about computer terminology, structure, terms. 

4 

More class time (i.e.) simulate FA mission processing. 

5 

Better understanding of terminology & F A 

6 

Going more detail on communications. 

7 

How every window operates. 

8 

Information with more PE. 

9 

Making the PE questions easier to understand. 

10 

Terminology 

11 

None 

12 


13 

Some of the classes were really boring. 1 don’t think there is any solution. 

14 

Hands on. 

15 

Same as above. 

16 

Understanding FA. 

17 

Strippers, or some entertainment. 

18 


19 

Experience/on the job training/ fielding. 

20 

No suggestions. 

21 

Specialized training for where we work at. 

22 

Better visual aids. 

23 

1 think it might help if they showed us just our MOS areas. 

24 

Better understanding of cannon. We are MLRS, lost many of us. 

25 

More interesting classes. 

26 

Time & experience/hands on. 

27 

Better understanding of Field Artillery. 

28 

It has a good set up. 

29 

Better understanding of the system. 

30 


31 


32 


33 

Maybe a smaller group to work with. 

34 

More hand-on work. 

35 

Nothing. 

36 

If each individual had their own computer. 

37 

More hand-on experience. 

38 

Getting more specified training on equipment dealing with my specific job and 


position. 

39 

Less window option. 

40 

Better understanding of how all of the different jobs in the Army work together. 

41 

Better understanding of Field Artillery. 

42 

Better understanding/more friendlier s/w with better prompts to continue. 

43 


44 


45 

More hands on training. 

46 

Windows that automatically go to next or corresponding windows. 

47 

Extend training on certain things and explain why it works and happens. 
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Case Summaries, Comment 3 



Other comments or questions that could help us develop supplemental training 

1 

2 

3 

4 | 

5 

6 

7 

More comfortable working environments; air conditioner/seats that don't hurt your!! 

/ ! 

8 

9 

10 

More P. E. time. 

11 


12 


13 

N/E 

14 

Ask me at the end of the course. 

15 


16 

AFATDS has too many windows, unnecessary windows. 

17 


18 


19 


20 

None. 

21 

Can you make the questions more clearly stated next time. 

22 


23 


24 

Need class geared towards MLRS, do not care about the cannon piece. Sometimes 
it seems people get cocky about the system, know it alls, thank you, 1st time these 


SM’s have seen this equipment. 

25 

Why do the teachers teach primarily toward cannon artillery when the students are all 
MLRS? We will never use most of the fields that are being reviewed. 

26 

Train only in specific areas of concern. 

27 


28 


29 


30 


31 


32 


33 


34 


35 


36 


37 


38 


39 


40 


41 


42 

Should be a separate window showing "required entries" for each type of mission or 
action. Required entries should be color coded. 

43 


44 


45 


46 


47 

Before leaving classroom, do a complete set up as it is in the field and simulate an 
exercise so we can completely understand the program and how it works on our level 


and not a made up unit that we are not. 
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